Although we have the two previous options, this section comes with its own USB camera driver implemented on top of OpenCV using the cv::VideoCapture class. It runs the camera and also allows you to change some of its parameters as long as they are supported by the camera's firmware. It also allows us to set the calibration information in the same way as with the FireWire cameras. With usb_cam, this is not possible because the CameraInfo message is not available. With regards to gscam, we have more control; we can change the camera configuration and see how to publish the camera's images and information in ROS. In order to implement a camera driver using OpenCV, we have two options on how we read images from the camera. First, we can poll with a target Frames Per Second (FPS); secondly, we can set a timer for the period of such FPS and, in the timer callback, we perform the actual reading. Depending on the FPS, one solution may be better than the other in terms of CPU consumption. Note that polling does not block anything, as the OpenCV reading function waits until an image is ready; meanwhile, other processes take the CPU. In general, for fast FPS, it is better to use polling so that we do not incur the time penalty of using the timer and its callback. For low FPS, the timer should be similar to polling and the code should be cleaner. We invite the reader to compare both implementations in the src/camera_polling.cpp and src/camera_timer.cpp files. For the sake of space, here we will only show you the timer-based approach. Indeed, the final driver in src/camera.cpp uses a timer. Note that the final driver also includes camera information management, which we will see in the sequel.

In the package, we must set the dependency with OpenCV, the ROS image message libraries, and related. They are the following packages:

<depend package="sensor_msgs"/> 
<depend package="opencv2"/> 
<depend package="cv_bridge"/> 
<depend package="image_transport"/> 

Consequently, in src/camera_timer.cpp, we have the following headers:

#include <ros/ros.h> 
#include <image_transport/image_transport.h> 
#include <cv_bridge/cv_bridge.h> 
#include <sensor_msgs/image_encodings.h> 
#include <opencv2/highgui/highgui.hpp> 

The image_transport API allows the publishing of images using several transport formats seamlessly, which can be compressed images, with different codecs, based on the plugins installed in the ROS system, for example, compressed and theora. The cv_bridge is used to convert OpenCV images to ROS image messages, for which we may need the image encoding of sensor_msgs in the case of grayscale/color conversion. Finally, we need the highgui API of OpenCV (opencv2) in order to use cv::VideoCapture.

Here, we will explain the main parts of the code in src/camera_timer.cpp,
which has a class that implements the camera driver. Its attributes are as follows:

ros::NodeHandle nh; 
image_transport::ImageTransport it; 
image_transport::Publisher pub_image_raw; 
 
cv::VideoCapture camera; 
cv::Mat image; 
cv_bridge::CvImagePtr frame; 
 
ros::Timer timer; 
 
intcamera_index; 
int fps; 

As usual, we need the node handle. Then, we need ImageTransport, which is used to send the images in all available formats in a seamless way. In the code, we only need to use Publisher (only one), but note that it must be a specialization of the image_transport library.

Then, we have the OpenCV stuff to capture images/frames. In the case of the frame, we directly use the cv_brigde frame, which is CvImagePtr, because we can access the image field it has.

Finally, we have the timer and the basic camera parameters for the driver to work. This is the most basic driver possible. These parameters are the camera index, that is, the number for the /dev/video? device, for example 0 for /dev/video0; the camera index is then passed to cv::VideoCapture. The fps parameter sets the camera FPS (if possible) and the timer. Here, we use an int value, but it will be a double in the final version, src/camera.cpp.

The driver uses the class constructor for the setup or initialization of the node,
the camera, and the timer:

nh.param<int>( "camera_index", camera_index, DEFAULT_CAMERA_INDEX 
 ); 
 
if ( not camera.isOpened() ) 
{ 
  ROS_ERROR_STREAM( "Failed to open camera device!" ); 
  ros::shutdown(); 
} 
 
nh.param<int>( "fps", fps, DEFAULT_FPS ); 
ros::Duration period = ros::Duration( 1. / fps ); 
 
pub_image_raw = it.advertise( "image_raw", 1 ); 
 
frame = boost::make_shared<cv_bridge::CvImage>(); 
frame->encoding = sensor_msgs::image_encodings::BGR8; 
 
timer = nh.createTimer( period, &CameraDriver::capture, this ); 

First, we open the camera and abort it if it does not open. Note that we must do this in the attribute constructor, shown as follows, where camera_index is passed by the parameter:

camera(camera_index ) 

Then, we read the fps parameter and compute the timer period, which is used to create the timer and set the capture callback at the end. We advertise the image publisher using the image transport API, for image_raw (RAW images), and initialize the frame variable.

The capture callback method reads and publishes images as follows:

camera>> frame->image; 
if( not frame->image.empty() ) 
{ 
  frame->header.stamp = ros::Time::now(); 
  pub_image_raw.publish( frame->toImageMsg() ); 
} 

The preceding method captures the images, checks whether a frame was actually captured, and in that case sets the timestamp and publishes the image, which is converted to a ROS image.

You can run this node with the following command:

    $ rosrun chapter9_tutorials camera_timer _camera_index:=0 _fps:=15  

This will open the /dev/video0 camera at 15 FPS.

Then, you can use image_viewer rqt_image_view to see the images. Similarly, for the polling implementation, you have the following command:

    $ roslaunch chapter5_tutorials camera_polling.launch
    camera_index:=0 fps:=15 view:=true  

With the previous command, you will see the /camera/image_raw topic images.

For the timer implementation, we also have the camera.launch file, which runs the final version and provides more options, which we will see throughout this entire chapter. The main contributions of the final version support for dynamic reconfiguration parameters and the provision of camera information, which includes camera calibration. We are going to show you how to do this in brief, and we advise that you look at the source code for a more detailed understanding.

As with the FireWire cameras, we can give support for the dynamic reconfiguration of the camera's parameters. However, most USB cameras do not support changing certain parameters. What we do is expose all OpenCV supported parameters and warn in case of error (or disable a few of them). The configuration file is in cfg/Camera.cfg; check it for details. It supports the following parameters:

• camera_index: This parameter is used to select the /dev/video? device
• frame_width and frame_height: These parameters give the image resolution
• fps: This parameter sets the camera FPS
• fourcc: This parameter specifies the camera pixel format in the FOURCC format (http://www.fourcc.org); the file format is typically YUYV or MJPEG, but they fail to change in most USB cameras with OpenCV
• brightness, contrast, saturation, and hue: These parameters set the camera's properties; in digital cameras, this is done by software during the acquisition process in the sensor or simply on the resulting image
• gain: This parameter sets the gain of the Analog-to-Digital Converter (ADC) of the sensor; it introduces salt-and-pepper noise into the image but increases the lightness in dark environments
• exposure: This parameter sets the lightness of the images, usually by adapting the gain and shutter speed (in low-cost cameras, this is simply the integration time of the light that enters the sensor)
• frame_id: This parameter is the camera frame and is useful if we use it for navigation, as we will see in the Using visual odometry with viso2 section
• camera_info_url: This parameter provides the path to the camera's information, which is basically its calibration

Then, in the following line of code in the driver, we use a dynamic reconfigure server:

#include <dynamic_reconfigure/server.h> 

We set a callback in the constructor:

server.setCallback( boost::bind( &CameraDriver::reconfig, this, 
 _1, _2 ) );

The setCallback constructor reconfigures the camera. We even allow it to change the camera and stop the current one when the camera_index changes. Then, we use the OpenCV cv::VideoCapture class to set the camera's properties, which are part of the parameters shown in the preceding line. As an example, in the case of frame_width, we use the following commands:

newconfig.frame_width = setProperty( camera, 
 CV_CAP_PROP_FRAME_WIDTH, newconfig.frame_width );

This relies on a private method named setProperty, which calls the set method of cv::VideoCapture and controls the cases in which it fails to print a ROS warning message. Note that the FPS has changed in the timer itself and cannot usually be modified in the camera. Finally, it is important to note that all this reconfiguration is done within a locked mutex to avoid acquiring any images while reconfiguring the driver.

In order to set the camera's information, ROS has a camera_info_manager library that helps us to do so, as shown in the following line:

#include <camera_info_manager/camera_info_manager.h> 

We use the library to obtain the CameraInfo message. In the capture callback of
the timer, we use image_transport::CameraPublisher, and not only for images. The code is as follows:

camera>> frame->image; 
if( not frame->image.empty() ) 
{ 
  frame->header.stamp = ros::Time::now(); 
 
  *camera_info = camera_info_manager.getCameraInfo(); 
  camera_info->header = frame->header; 
 
  camera_pub.publish( frame->toImageMsg(), camera_info ); 
} 

This is run within the mutex mentioned previously for the reconfiguration method. We do the same as for the first version of the driver but also retrieve the camera information from the manager, which is set with the node handler, the camera name, and amera_info_url in the reconfiguration method (which is always called once on loading). Then, we publish both the image/frame (ROS image) and the CameraImage messages.

In order to use this driver, use the following command:

    $ roslaunch chapter5_tutorials camera.launch view:=true  

The command will use the config/camera/webcam.yaml parameter as default, which sets all the dynamic reconfiguration parameters seen so far.

You can check that the camera is working with rostopic list and rostopic hz/camera/image_raw; you can also check with image_view or rqt_image_view.

With the implementation of this driver, we have used all the resources available in ROS to work with cameras, images, and Computer Vision. In the following sections, for the sake of clarity, we will explain each of them separately.