In the previous chapter, you learned how to detect and track an object of interest in the video stream of a webcam, even if the object is viewed from different angles or distances, or under partial occlusion. Here, we will take the tracking of interesting features a step further and see what we can learn about the entire visual scene by studying similarities between image frames.

The goal of this chapter is to study how to reconstruct a scene in 3D by inferring the geometrical features of the scene from camera motion. This technique is sometimes referred to as structure from motion. By looking at the same scene from different angles, we will be able to infer the real-world 3D coordinates of different features in the scene. This process is known as triangulation, which allows us to reconstruct the scene as a 3D point cloud.

If we take two pictures of the same scene from different angles, we can use feature matching or optic flow to estimate any translational and rotational movement that the camera underwent between taking the two pictures. However, in order for this to work, we will first have to calibrate our camera.

In this chapter, we will cover the following topics:

• Learning about camera calibration
• Setting up the app
• Estimating the camera motion from a pair of images
• Reconstructing the scene
• Understanding 3D point cloud visualization
• Learning about structure from motion

Once you complete the app, you will understand the classical approaches that are used to make a 3D reconstruction of a scene or object given several images taken from different view points. You will be able to apply these approaches in your own apps related to constructing 3D models from camera images or videos.