In the previous section, we dealt with 2D object recognition using a 2D and 3D sensor. In this section, we will discuss 3D recognition. So what is 3D object recognition? In 3D object recognition, we take the 3D data or point cloud data of the surroundings and 3D model of the object. Then, we match the scene object with the trained model, and if there is a match found, the algorithm will mark the area of detection.

In real-world scenarios, 3D object recognition/detection is much better than 2D because in 3D detection, we use the complete information of the object, similar to human perception. But there are many challenges involved in this process too. Some of the main constrains are computational power and expensive sensors. We may need more expensive computers to process 3D information; also, the sensors for this purpose are costlier.

Some of the latest applications using 3D object detection and recognition are autonomous robots, especially self-driving cars. Self-driving cars have a LIDAR such as Velodyne (http://velodynelidar.com/) that can provide a complete 3D point cloud around the vehicle. The computer inside takes the 3D input and run various detectors to find pedestrians, cyclists, and other obstacles for a collision-free ride.

Like we discussed in the beginning, in the Amazon Picking Challenge and other such applications, the picking and placing needs 3D recognition capability. The following figure shows how an autonomous car perceives the world. The data shown around the car is the 3D point cloud, which helps it detect objects and predict a collision-free route.

3D object recognition has many applications, and in this section, you are going to see how to perform a basic 3D object recognition using ROS and cheap depth sensors.