Motion and Structure from Image Sequences

Motion and Structure from Image Sequences is invaluable reading for researchers, graduate students, and practicing engineers dealing with computer vision. It presents a balanced treatment of the theoretical and practical issues, including very recent results. The rich collection of algorithms, applicable to monocular or stereo views, makes this book particularly attractive to practitioners.

1. Introduction.- 1.1 Formulation of the Problem.- 1.2 Overview of Recent Progress.- 1.3 Bibliographical Notes.- 2. Image Matching.- 2.1 Approaches to Establishing Correspondences.- 2.2 An Approach to Image Matching.- 2.2.1 Image Attributes.- 2.2.2 Relationships of the Attributes.- 2.2.3 Intra-Regional Smoothness and Occlusion.- 2.2.4 Multi-Resolution Multi-Grid Structure.- 2.2.5 Limitations.- 2.3 Algorithm.- 2.3.1 Locally Rigid Motion.- 2.3.2 3-D Motion and Image Plane Motion.- 2.3.3 Motion Insensitive Image Attributes.- 2.3.4 Edgeness and Cornerness.- 2.3.5 Smoothness.- 2.3.6 Minimizing Residuals.- 2.3.7 Recursive Blurring.- 2.3.8 Preprocessing and Normalization.- 2.3.9 Outline of the Matching Algorithm.- 2.4 Motion and Structure Computation.- 2.5 Refinements.- 2.5.1 Bottom-Up and Top-Down.- 2.5.2 Refinement Based on Rigidity.- 2.6 Examples.- 2.7 Summary.- 2.8 Bibliographical Notes.- 3. Two-View Analysis.- 3.1 Some Basic Issues.- 3.2 An Algorithm.- 3.2.1 Problem Statement.- 3.2.2 Algorithm.- 3.2.3 Justification of the Algorithm.- 3.3 Error Estimation.- 3.3.1 Algorithm and Perturbation.- 3.3.2 Perturbation of Eigenvalues and Eigenvectors.- 3.3.3 Error Estimation for the Algorithm.- 3.4 Error Analysis.- 3.4.1 Structure of the Scene.- 3.4.2 Motion Parameters.- 3.4.3 System Parameters.- 3.5 Performance.- 3.5.1 Performance.- 3.5.2 Error Estimation.- 3.5.3 Error versus Motion and System Parameters.- 3.5.4 Real Images.- 3.6 Summary.- 3.7 Bibliographical Notes.- Appendix 3.A Perturbation of Eigenspace.- Appendix 3.B Quaternions.- Appendix 3.C Alternative Fitting.- 4. Optimization.- 4.1 Motivations.- 4.2 Stability of Linear Algorithms.- 4.2.1 The Linear Algorithms.- 4.2.2 The Epipolar Constraint.- 4.2.3 Using the Constraint in Matrix E.- 4.2.4 A Type of Motion.- 4.2.5 Beyond the Epipolar Constraint.- 4.3 Maximum Likelihood Estimation.- 4.3.1 Gaussian Distribution.- 4.3.2 Uncertainty Polyhedron Distribution.- 4.4 A Two-Step Approach and Computation.- 4.5 Minimum Variance Estimation.- 4.6 Error Estimation and Error Bounds.- 4.6.1 Error Estimation.- 4.6.2 Error Bounds.- 4.7 Batch and Sequential Methods.- 4.7.1 Linear Systems.- 4.7.2 Nonlinear Systems.- 4.7.3 Recursive-Batch Solution.- 4.8 Numerical Examples.- 4.8.1 Whether a Good Initial Guess Is Necessary.- 4.8.2 Epipolar Improvement versus Optimization.- 4.8.3 Sequential versus Batch Solutions.- 4.8.4 Uncertainty Polyhedron Model.- 4.8.5 Error Estimation and Error Bound.- 4.8.6 Inherent Limitation of Optical Flow.- 4.8.7 Real World Images.- 4.9 Summary.- 4.10 Bibliographical Notes.- Appendix 4.A Decomposability.- Appendix 4.B Weights.- Appendix 4.C Triangulation with Noise.- Appendix 4.D Matrix Derivatives.- Appendix 4.E Cramér-Rao Bound.- 5. Planar Scenes.- 5.1 Planar Scene as a Degenerate Case.- 5.2 Motion from a Plane.- 5.2.1 Problem Statement.- 5.2.2 Algorithm Overview.- 5.2.3 Intermediate Parameter Matrix F.- 5.2.4 General Case.- 5.2.5 Two Solutions Are Both Valid.- 5.2.6 Special Cases.- 5.3 Inherent Uniqueness.- 5.3.1 Decomposability.- 5.3.2 Rank Condition Is Algorithm-Independent.- 5.3.3 The Fundamental Theorem.- 5.3.4 Plane-Perceivable Surfaces.- 5.3.5 Optimality.- 5.3.6 Three-View Problem.- 5.4 Examples.- 5.4.1 Simulations.- 5.4.2 Real Images.- 5.5 Conclusions.- 5.6 Bibliographical Notes.- Appendix 5.A Conditions on the Rank of A.- Appendix 5.B Two Solutions from F.- Appendix 5.C Plane-Perceivable Surfaces.- Appendix 5.D Condition on Consistent Normals.- Appendix 5.E Two-View Algorithm.- Appendix 5.F Error Estimation.- 6. From Line Correspondences.- 6.1 Lines as Features.- 6.2 Solution and Algorithm.- 6.2.1 Why Two Views Are Not Sufficient.- 6.2.2 From Three Views.- 6.2.3 Two Important Equations.- 6.2.4 A Geometrical View.- 6.2.5 Intermediate Parameters.- 6.2.6 Motion from Intermediate Parameters.- 6.2.7 Structure and Sign of Translation Vectors.- 6.2.8 In the Presence of Noise.- 6.2.9 Algorithm.- 6.3 Degeneracy.- 6.4 Optimization.- 6.4.1 Lines from Pixels.- 6.4.2 Optimal Solution.- 6.4.3 How Accurate It Can Be.- 6.5 Simulations.- 6.5.1 Setup.- 6.5.2 Linear Algorithm.- 6.5.3 Algorithm with Optimization.- 6.5.4 Comparison with the Bound.- 6.5.5 Comparison with the Point-Based Algorithm.- 6.6 Conclusions and Discussions.- 6.7 Bibliographical Notes.- Appendix 6.A Ranks.- Appendix 6.B Unique Consistent Assignment.- Appendix 6.C Degeneracy.- Appendix 6.D Distinct Locations Are Necessary.- Appendix 6.E Alternative Degeneracy Condition.- 7. Stereo.- 7.1 Stereo Camera Systems.- 7.2 Stereo Triangulation.- 7.3 Closed-Form Solution.- 7.3.1 A Matrix-Weighted Objective Function.- 7.3.2 Unweighted and Scalar-Weighted Versions.- 7.3.3 Matrix-Weighted Least-Squares Solution.- 7.3.4 Uniqueness.- 7.4 Iterative Optimal Solution.- 7.4.1 A Basic Objective Function.- 7.4.2 Optimization Using Space Decomposition.- 7.4.3 Estimating Errors.- 7.5 Outliers and Robust Estimators.- 7.5.1 Some Basic Concepts of Robust Statistics.- 7.5.2 A Robust Method for Motion Estimation.- 7.5.3 Using Closed-Form Solution.- 7.6 Examples.- 7.6.1 Simulations.- 7.6.2 Experiments with a Real Stereo Setup.- 7.7 Without Stereo Correspondences.- 7.7.1 Motivations.- 7.7.2 Motions Seen from Two Cameras.- 7.7.3 Determining Complete Motion and Structure.- 7.7.4 Computational Aspect of the Degeneracy.- 7.8 Long Image Sequences.- 7.8.1 Local and Global Coordinate Systems.- 7.8.2 A Recursive-Batch Approach.- 7.8.3 Recursive-Batch Updating.- 7.8.4 Local and Global Representations.- 7.8.5 Numerical Examples.- 7.9 Conclusions.- 7.10 Bibliographical Notes.- Appendix 7.A MWCC Theorem.- Appendix 7.B Least-Squares Matrix Fitting.- 8. Motion Modeling and Prediction.- 8.1 Coherence of Motion.- 8.2 The LCAM Model.- 8.2.1 Motion of a Rigid Body in 3-D.- 8.2.2 Motion of Rotation Center.- 8.2.3 Solutions of the Coefficient Equation.- 8.2.4 Continuous and Discrete Motions.- 8.3 Estimation and Prediction.- 8.4 Monocular Vision.- 8.5 Optimization.- 8.6 Experimental Examples.- 8.6.1 Computer Simulation.- 8.6.2 Experiment with Real Images.- 8.7 Summary.- 8.8 Bibliographical Notes.- Appendix 8.A Solution of Coefficient Equations.- Appendix 8.B Rotation without Precession.- Appendix 8.C Singularity of the Matrix.- References.