The goal of this chapter is to develop a number of image processing filters and apply them to the video stream of a webcam in real time. These filters will rely on various OpenCV functions to manipulate matrices through splitting, merging, arithmetic operations, and applying lookup tables for complex functions.
The three effects are as follows:
- Black-and-white pencil sketch: To create this effect, we will make use of two image blending techniques, known as dodging and burning
- Warming/cooling filters: To create these effects, we will implement our own curve filters using a lookup table
- Cartoonizer: To create this effect, we will combine a bilateral filter, a median filter, and adaptive thresholding
OpenCV is such an advanced toolchain that often the question is not how to implement something from scratch, but rather which pre-canned implementation to choose for your needs. Generating complex effects is not hard if you have a lot of computing resources to spare. The challenge usually lies in finding an approach that not only gets the job done, but also gets it done in time.
Instead of teaching the basic concepts of image manipulation through theoretical lessons, we will take a practical approach and develop a single end-to-end app that integrates a number of image filtering techniques. We will apply our theoretical knowledge to arrive at a solution that not only works but also speeds up seemingly complex effects so that a laptop can produce them in real time.
The following screenshot shows the final outcome of the three effects running on a laptop:
Note
All of the code in this book is targeted for OpenCV 2.4.9 and has been tested on Ubuntu 14.04. Throughout this book, we will make extensive use of the NumPy package (http://www.numpy.org). In addition, this chapter requires the UnivariateSpline module of the SciPy package (http://www.scipy.org) as well as the wxPython 2.8 graphical user interface (http://www.wxpython.org/download.php) for cross-platform GUI applications. We will try to avoid further dependencies wherever possible.
The final app will consist of the following modules and scripts:
filters: A module comprising different classes for the three different image effects. The modular approach will allow us to use the filters independently of any graphical user interface (GUI).filters.PencilSketch: A class for applying the pencil sketch effect to an RGB color image.filters.WarmingFilter: A class for applying the warming filter to an RGB color image.filters.CoolingFilter: A class for applying the cooling filter to an RGB color image.filters.Cartoonizer: A method for applying the cartoonizer effect to an RGB color image.gui: A module that provides a wxPython GUI application to access the webcam and display the camera feed, which we will make extensive use of throughout the book.gui.BaseLayout: A generic layout from which more complicated layouts can be built.chapter1: The main script for this chapter.chapter1.FilterLayout: A custom layout based ongui.BaseLayoutthat displays the camera feed and a row of radio buttons that allows the user to select from the available image filters to be applied to each frame of the camera feed.chapter1.main: The main function routine for starting the GUI application and accessing the webcam.