In this chapter, we explored a way to label the potentially interesting objects in a visual scene, even if their shape and number is unknown. We explored natural image statistics using Fourier analysis, and implemented a state-of-the-art method for extracting the visually salient regions in the natural scenes. Furthermore, we combined the output of the salience detector with a tracking algorithm to track multiple objects of unknown shape and number in a video sequence of a soccer game.

It would now be possible to extend our algorithm to feature more complicated feature descriptions of proto-objects. In fact, mean-shift tracking might fail when the objects rapidly change size, as would be the case if an object of interest were to come straight at the camera. A more powerful tracker, which comes for free in OpenCV, is cv2.CamShift. CAMShift stands for Continuously Adaptive Mean-Shift, and bestows upon mean-shift the power to adaptively change the window size. Of course, it would also be possible to simply replace the mean-shift tracker with a previously studied technique such as feature matching or optic flow.

In the next chapter, we will move to the fascinating field of machine learning, which will allow us to build more powerful descriptors of objects. Specifically, we will focus on both detecting (where?) and identifying (what?) the street signs in images. This will allow us to train a classifier that could be used in a dashboard camera in your car, and will familiarize us with the important concepts of machine learning and object recognition.