As a computer vision developer, there is absolutely no way you can avoid dealing with video feeds from stored video files or cameras and other such sources. Treating video frames as individual images is one way to process videos, which surprisingly doesn't require much more effort or knowledge of the algorithms than what you have learned so far. For instance, you can apply a smoothening filter on a video, or in other words, a set of video frames, the same way as you would when you apply it on an individual image. The only trick here is that you must extract each frame from a video, as described in Chapter 2, Getting Started with OpenCV. However, in computer vision, there are certain algorithms that are meant to work with consecutive video frames and the result on their operation depends not just on an individual image but also on the result of the same operation on the previous frames. Both of the algorithm types we just mentioned will be the main topics covered in this chapter.

After learning about histograms and back-projection images in the previous chapter, we are ready to take on computer vision algorithms that are used to detect and track objects in real-time. These algorithms highly rely on a firm understanding on all the topics we learned in Chapter 5, Back-Projection and Histograms. Based on this, we'll start this chapter with a couple of simple examples about how to use the computer vision algorithms we've learned so far, to process frames from a video file or camera, and then we'll move on to learn about two of the most famous object detection and tracking algorithms, the Mean Shift and CAM Shift algorithms. Then, we'll learn how to use the Kalman filter to correct the result of our object detection and tracking algorithms and how to remove noise from the results to get a better tracking result. We'll end this chapter by learning about motion analysis and background/foreground extraction.

In this chapter, we'll cover the following:

• How to apply filters and perform such operations on videos
• Using the Mean Shift algorithm to detect and track objects
• Using the CAM Shift algorithm to detect and track objects
• Using the Kalman filter to improve tracking results and remove noise
• Using the background and foreground extraction algorithms