Open Source Computer Vision (OpenCV) is a library that has APIs to perform computer vision applications. The project was started in Intel Russia, and later on, it was supported by Willow Garage and Itseez. In 2016, Itseez was acquired by Intel.
OpenCV is a cross-platform library that supports most operating systems. Now, it also has an open source BSD license, so we can use it for research and commercial applications. The OpenCV version interfaced with ROS Kinetic is 3.1. The 3.x versions of OpenCV have a few changes to the APIs from the 2.x versions.
The OpenCV library is integrated into ROS through a package called vision_opencv.
The vision_opencv metapackage has two packages:
- cv_bridge: This package is responsible for converting the OpenCV image data type (cv::Mat) into ROS Image messages (sensor_msgs/Image.msg).
- image_geometry: This package helps us interpret images geometrically. This node will aid in processing such as camera calibration and image rectification.
Out of these two packages, we are mainly dealing with cv_bridge. Using cv_bridge, the face tracker node can convert ROS Image messages from usb_cam to the OpenCV equivalent, cv::Mat. After converting to cv::Mat, we can use OpenCV APIs to process the camera image.
Here is a block diagram that shows the role of cv_bridge in this project:
Here, cv_bridge is working between the usb_cam node and face-tracking node. We'll learn more about the face-tracking node in the next section. Before that, it will be good if you get an idea of its working.
Another package we are using to transport ROS Image messages between two ROS nodes is image_transport (http://wiki.ros.org/image_transport). This package is always used to subscribe to and publish image data in ROS. The package can help us transport images in low bandwidth by applying compression techniques. This package is also installed along with the full ROS desktop installation.
That's all about OpenCV and the ROS interface. In the next section, we are going to work with the first package of this project: face_tracker_pkg.