The great advantage of doing Computer Vision in ROS is the fact that we do not have to re-invent the wheel. A lot of third-party software is available, and we can also connect our vision stuff to the real robots or perform simulations. Here, we are going to enumerate interesting Computer Vision tools for the most common visual tasks, but we will only explain one of them in detail, including all the steps to set it up.

We will do this for visual odometry, but other packages are also easy to install and it is also easy to start playing with them; simply follow the tutorials or manuals in the links provided here:

• Visual Servoing: Also known as vision-based robot control, this is a technique that uses feedback information obtained from a vision sensor to control the motion of a robot, typically an arm used for grasping. In ROS, we have a wrapper of the Visual Servoing Platform (ViSP) software (http://www.irisa.fr/lagadic/visp/visp.html and http://www.ros.org/wiki/vision_visp). ViSP is a complete cross-platform library that allows you to prototype and develop applications in visual tracking and visual serving. The ROS wrapper provides a tracker that can be run with the visp_tracker (moving edge tracker) node as well as visp_auto_tracker (a model-based tracker). It also helps to calibrate the camera and perform hand-to-eye calibration, which is crucial for visual serving in grasping tasks.
• Augmented Reality: An Augmented Reality (AR) application involves overlaying virtual imagery in the real world. A well-known library for this purpose is ARToolkit (https://www.hitl.washington.edu/artoolkit/). The main problem in this application is tracking the user's viewpoint to draw the virtual imagery on the viewpoint where the user is looking in the real world. ARToolkit video tracking libraries calculate the real camera position and orientation relative to physical markers in real time. In ROS, we have a wrapper named ar_pose (http://www.ros.org/wiki/ar_pose). It allows us to track single or multiple markers where we can render our virtual imagery (for example, a 3D model).
• Perception and object recognition: Most basic perception and object recognition is possible with the OpenCV libraries. However, there are several packages that provide an object recognition pipeline, such as the object_recognition stack, which provides tabletop_object_detector to detect objects on a table; for example, a more general solution provided by Object Recognition Kitchen (ORK) can be found at http://wg-perception.github.io/object_recognition_core. It is also worth mentioning a tool called RoboEarth (http://www.roboearth.org), which allows you to detect and build 3D models of physical objects and store them in a global database accessible for any robot (or human) worldwide. The models stored can be 2D or 3D and can be used to recognize similar objects and their viewpoint, that is, to identify what the camera/robot is watching. The RoboEarth project is integrated into ROS, and many tutorials are provided to have a running system (http://www.ros.org/wiki/roboearth), although it does not support officially the latest versions of ROS.
• Visual odometry: A visual odometer is an algorithm that uses the images of the environment to track features and estimate the robot's movement, assuming a static environment. It can solve the six DoF poses of the robot with a monocular or stereo system, but it may require additional information in the monocular case. There are two main libraries for visual odometry: libviso2 (http://www.cvlibs.net/software/libviso2.html) and libfovis (http://www.ros.org/wiki/fovis_ros), both of which have wrappers for ROS. The wrappers just expose these libraries to ROS. They are the viso2 and fovis packages, respectively. In the following section, we will see how we can do visual odometry with our homemade stereo camera using the viso2_ros node of viso2. We also show how to install them, which at the moment needs to be done from source because these packages do not support ROS Kinetic officially; however, there is no other alternative for them integrated in ROS. The libviso2 library allows us to do monocular and stereo visual odometry. However, for monocular odometry, we also need the pitch and heading for the ground plane estimation. You can try the monocular case with one camera and an IMU , but you will always have better results with a good stereo pair, correctly calibrated, as seen so far in this chapter. Finally, libfovis does not allow the monocular case, but it does support RGBD cameras, such as the Kinect sensor . In regards the stereo case, it is possible to try both libraries and see which one works better in your case. Here, we will give you a step-by-step tutorial on how to install and run viso2 in ROS and fovis with Kinect.