3.11. Conclusion

In this chapter, the relations between multiple views of a 3D scene were discussed. The discussion started by introducing the basic concepts of projective geometry and tensor calculus. Then, the pinhole camera model, the epipolar geometry, and the multiple view tensors were discussed. Next, approaches that rely on those concepts to recover both the structure and motion from a sequence of images were presented and illustrated with some real-world examples and applications.

Acknowledgment

The authors wish to acknowledge the financial support of the EU projects InViews, Attest, Vibes, and Murale, as well as the contributions of their (former) colleagues in Leuven and Lund, where most of the presented work was carried out. Luc Van Gool, Maarten Vergauwen, Frank Verbiest, Kurt Cornells, Jan Tops, Reinhard Koch, and Benno Heigl have contributed to generate the results that illustrate this chapter.

Bibliography

[1] S. Avidan and A. Shashua . Novel view synthesis in tensor space. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pages 1034–1040, 1997.

[2] N. Ayache . Artificial Vision for Mobile Robots: Stereo Vision and Multi-sensory Perception. MIT Press, 1991.

[3] C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen . Unstructured lumigraph rendering. In Proc. SIGGRAPH, 2001.

[4] S. Chen and L. Williams . View interpolation for image synthesis. Computer Graphics, 27 (Annual Conference Series): 279-288, 1993.

[5] O. Chum and J. Matas . Randomized ransac and t(d,d) test. In Proc. British Machine Vision Conference, 2002.

[6] K. Cornells, M. Pollefeys, M. Vergauwen, and L. Van Gool . Augmented reality from uncalibrated video sequences. In A. Zisserman, M. Pollefeys, L. Van Gool, and A. Fitzgibbon (Eds.), 3D Structure from Images-SMILE 2000, volume 2018, pages 150-167. Springer-Verlag, 2001.

[7] K. Cornells, M. Pollefeys, M. Vergauwen, F. Verbiest, and L. Van Gool . Tracking based structure and motion recovery for augmented video productions. In Proceedings ACM Symposium on Virtual Reality Software and TechnologyVRST 2001, pages 17–24, November 2001.

[8] I. Cox, S. Hingoraini, and S. Rao . A maximum likelihood stereo algorithm. Computer Vision and Image Understanding, 63(3): 542–567, May 1996.

[9] H. S.M. Coxeter . Protective Geometry. Blaisdell Publishing Company, 1964.

[10] B. Curless and M. Levoy . A volumetric method for building complex models from range images. In Proc. SIGGRAPH '96, pages 303-312, 1996.

[11] L. de Agapito, R. Hartley, and E. Hayman . Linear selfcalibration of a rotating and zooming camera. In Proc. IEEE Conf. Comp. Vision Patt. Recog., volume I, pages 15–21, June 1999.

[12] L. Falkenhagen . Depth estimation from stereoscopic image pairs assuming piecewise continuous surfaces. In Proc. of European Workshop on Combined Real and Synthetic Image Processing for Broadcast and Video Production, November 1994.

[13] O. Faugeras . What can be seen in three dimensions with an uncalibrated stereo rig? In Proc. European Conf. on Computer Vision, pages 563-578, 1992.

[14] O. Faugeras . Three-Dimensional Computer Vision. MIT Press, 1993.

[15] O. Faugeras and Q.-T. Luong . The Geometry of Multiple Images. MIT Press, 2001.

[16] O. Faugeras, Q.-T. Luong, and S. Maybank . Camera self-calibration: Theory and experiments. In Computer Vision-ECCV'92, volume 588 of Lecture Notes in Computer Science, pages 321-334. Springer-Verlag, 1992.

[17] M. Fischler and R. Bolles . Random sampling consensus: A paradigm for model fitting with application to image analysis and automated cartography. Commun. Assoc. Comp. Mach., 24: 381–395, 1981.

[18] S. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen . The Lumigraph. In Proc. SIGGRAPH '96, pages 43–54. ACM Press, New York, 1996.

[19] C. Harris and M. Stephens . A combined corner and edge detector. In Fourth Alvey Vision Conference, pages 147-151, 1988.

[20] R. Hartley . An algorithm for self calibration from several views. In Proc. IEEE Conf. Comp. Vision Patt. Recog., pages 908-912, 1994.

[21] R. Hartley . Euclidean reconstruction from uncalibrated views. In J. L. Mundy, A. Zisserman, and D. Forsyth (Eds.), Applications of Invariance in Computer Vision, volume 825 of Lecture Notes in Computer Science, pages 237-256. Springer-Verlag, 1994.

[22] R. Hartley . In defense of the eight-point algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(6): 580–593, June 1997.

[23] R. Hartley and P. Sturm . Triangulation. Computer Vision and Image Understanding, 68(2): 146–157, 1997.

[24] R. Hartley and A. Zisserman . Multiple View Geometry in Computer Vision. Cambridge University Press, 2000.

[25] B. Heigl, R. Koch, M. Pollefeys, J. Denzler, and L. Van Gool . Plenoptic modeling and rendering from image sequences taken by hand-held camera. In Proc. DAGM, pages 94–101, 1999.

[26] A. Heyden . Geometry and Algebra of Multiple Projective Transformations. PhD thesis, Lund University, Sweden, 1995.

[27] A. Heyden . Tensorial properties of multilinear constraints. Mathematical Methods in the Applied Sciences, 23: 169–202, 2000.

[28] A. Heyden and K. Åström . Euclidean reconstruction from constant intrinsic parameters. In Proc. 13th International Conference on Pattern Recognition, pages 339-343. IEEE Computer Soc. Press, 1996.

[29] A. Heyden and K. Åström . Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pages 438-443. IEEE Computer Soc. Press, 1997.

[30] A. Heyden and K. Åström . Algebraic properties of multilinear constraints. Mathematical Methods in the Applied Sciences, 20: 1135–1162, 1997.

[31] F. Kahl . Critical motions and ambiguous euclidean reconstructions in autocalibration. In Proc. International Conference on Computer Vision, pages 469-475, 1999.

[32] F. Kahl, B. Triggs, and K. Åström . Critical motions for autocalibration when some intrinsic parameters can vary. Journal of Mathematical Imaging and Vision, 13(2): 131–146, 2000.

[33] R. Koch, B. Heigl, and M. Pollefeys . Image-based rendering from uncalibrated lightfields with scalable geometry. In G. Gimel'farb, R. Klette, and T. Huang (Eds.), Multi-Image Analysis, volume 2032 of Lecture Notes in Computer Science, pages 51–66. Springer-Verlag, 2001.

[34] R. Koch, M. Pollefeys, and L. Van Gool . Multi viewpoint stereo from uncalibrated video sequences. In Proc. European Conference on Computer Vision, pages 55–71, 1998.

[35] R. Koch, M. Pollefeys, B. Heigl, L. Van Gool, and H. Niemann . Calibration of hand-held camera sequences for plenoptic modeling. In Proc. International Conference on Computer Vision, pages 585-591, 1999.

[36] S. Laveau and O. Faugeras . Oriented projective geometry for computer vision. In B. Buxton and R. Cipolla (Eds.), Proc. European Conference on Computer Vision, Lecture Notes in Computer Science, Vol. 1064, pages 147-156. Springer-Verlag, 1996.

[37] M. Levoy and P. Hanrahan . Lightfield rendering. In Proc. SIGGRAPH '96, pages 31–42. ACM Press, 1996.

[38] H. C. Longuet-Higgins . A computer algorithm for reconstructing a scene from two projections. Nature, 293: 133–135, 1981.

[39] D. Lowe . Object recognition from local scale-invariant features. In Proc. International Conference on Computer Vision, pages 1150–1157, 1999.

[40] J. Matas, S. Obdrzalek, and O. Chum . Local affine frames for wide-baseline stereo. In Proc. 16th International Conference on Pattern Recognition, volume 4, pages 363-366, 2002.

[41] G. Van Meerbergen, M. Vergauwen, M. Pollefeys, and L. Van Gool . A hierarchical symmetric stereo algorithm using dynamic programming. International Journal on Computer Vision, 47(l/2/3): 275–285, 2002.

[42] N. Myklestad . Cartesian Tensors-the Mathematical Language of Engineering. Van Nostrand, Princeton, 1967.

[43] M. Pollefeys . Self-calibration and metric 3D reconstruction from uncalibrated image sequences. PhD thesis, Katholieke Universiteit Leuven, 1999.

[44] M. Pollefeys and L. Van Gool . Stratified self-calibration with the modulus constraint. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8):707–724, 1999.

[45] M. Pollefeys and L. Van Gool . Do ambiguous reconstructions always give ambiguous images? In Proc. International Conference on Computer Vision, pages 187-192, 2001.

[46] M. Pollefeys, R. Koch, and L. Van Gool . Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Proc. International Conference on Computer Vision, pages 90–95. Narosa Publishing House, 1998.

[47] M. Pollefeys, R. Koch, and L. Van Gool . Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. International Journal of Computer Vision, 32(1): 7–25, 1999.

[48] M. Pollefeys, R. Koch, and L. Van Gool . A simple and efficient rectification method for general motion. In Proc. International Conference on Computer Vision, pages 496-501, 1999.

[49] M. Pollefeys, F. Verbiest, and L. Van Gool . Surviving dominant planes in uncalibrated structure and motion recovery. In A. Heyden, G. Sparr, M. Nielsen, and P. Johansen (Eds.), 7th European Conference on Computer Vision, volume 2351 of Lecture Notes in Computer Science, pages 837-851, 2002.

[50] W. Press, S. Teukolsky, and W. Vetterling . Numerical Recipes in C. The Art of Scientific Computing. Cambridge University Press, 1992.

[51] P. Rousseeuw . Robust Regression and Outlier Detection. Wiley, 1987.

[52] D. Scharstein and R. Szeliski . A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1/2/3): 7–42, April-June 2002.

[53] C. Schmid and R. Mohr . Local grayvalue invariants for image retrieval. IEEE PAMI, 19(5): 530–534, 1997.

[54] S. Seitz and C. Dyer . View morphing. Computer Graphics, Annual Conference Series, 30: 21–30, 1996.

[55] A. Shashua . Trilinearity in visual recognition by alignment. In Proc. European Conf. on Computer Vision, 1994.

[56] J. Shi and C. Tomasi . Good features to track. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994.

[57] C. Slama . Manual of Photogrammetry. 4th ed., American Society of Photogrammetry, USA, 1980.

[58] B. Spain . Tensor Calculus. University Mathematical Texts, Oliver and Boyd, Edingburgh, 1953.

[59] G. Sparr . Simultaneous reconstruction of scene structure and camera locations from uncalibrated image sequences. In Proc. International Conf. on Pattern Recognition, 1996.

[60] M. E. Spetsakis and J. Aloimonos . A unified theory of structure from motion. In Proc. DARPA IU Workshop, 1990.

[61] P. Sturm . Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction. In Proc. Conference on Computer Vision and Pattern Recognition, pages 1100–1105. IEEE Computer Society Press, 1997.

[62] P. Sturm . Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length. In Proc. 10th British Machine Vision Conference, pages 63–72, 1999.

[63] P. Sturm . Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length. In Proc. BMVC, 1999.

[64] B. Triggs . Matching constraints and the joint image. In Proc. Int. Conf. on Computer Vision, 1995.

[65] B. Triggs . The absolute quadric. In Proc. 1997 Conference on Computer Vision and Pattern Recognition, pages 609-614. IEEE Computer Society Press, 1997.

[66] B. Triggs . Autocalibration from planar scenes. In ECCV, volume I, pages 89–105, June 1998.

[67] B. Triggs, P. McLauchlan, R. Hartley, and A. Fiztgibbon . Bundle adjustment-A modern synthesis. In R. Szeliski, B. Triggs, A. Zisserman (Eds.), Vision Algorithms: Theory and Practice, LNCS Vol. 1883, pages 298-372. Springer-Verlag, 2000.

[68] G. Turk and M. Levoy . Zippered polygon meshes from range images. In Proceedings of SIGGRAPH '94, pages 311-318, 1994.

[69] T. Tuytelaars and L. Van Gool . Wide baseline stereo based on local, affinely invariant regions. In British Machine Vision Conference, pages 412-422, 2000.

[70] Z. Zhang, R. Deriche, O. Faugeras, and Q.-T. Luong . A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence Journal, 78: 87–119, October 1995.