This project can be divided into two sections: teleoperation using Leap Motion and streaming images to an Android phone to get a VR experience inside a VR headset. Before going to discuss each design aspect, let's see how we have to interconnect these devices.

The following figure shows how the components are interconnected for this project:

You can see that each device (that is, PC and Android phone) is connected to a Wi-Fi router, and the router has assigned an IP to each device. Each device communicates using these IP addresses. You will see the importance of these addresses in the upcoming sections.

Next, we will see how we can teleoperate a robot in ROS using Leap Motion. We will be controlling it while wearing the VR headset. So, we don't need to press any buttons to move the robot; rather, we can just move it with our hands.

The basic operation involved here is converting the Leap Motion data into ROS Twist messages. Here, we are only interested in reading the orientation of the hand. We are taking roll, pitch, and yaw and mapping them into ROS Twist messages. Here is how:

The preceding figure shows how Leap Motion data is manipulated into ROS Twist messages. The Leap Motion PC Driver/SDK interfaces the controller with Ubuntu, and the Leap Motion ROS Driver, which works on top of this driver/SDK, fetches the hand and finger position and publishes it as ROS topics. The node we are going to write can convert the hand position to Twist data, which will subscribe to the Leap Motion data topic called /leapmotion/data, convert it into corresponding command velocities, and publish to the topic called /cmd_vel_mux/input/teleop. The conversion algorithm is based on comparing the hand orientation value. If the value is in a particular range, we will publish a particular Twist value.

Here is the simple algorithm that converts Leap Motion orientation data into Twist messages:

1. Take the orientation values of hand, such as yaw, pitch, and roll, from the Leap Motion ROS driver.
2. The roll movement of the hand corresponds to robot rotation. If the hand rotates anticlockwise, then the robot will be triggered to rotate anticlockwise by sending a command velocity. This will be the opposite case of a roll of the hand in the clockwise direction.
3. If the hand is pitched down, the robot will move forward, and if the hand is pitched up, the robot will move backward.
4. If there is no hand movement, the robot will stop.

This is a simple algorithm to move a robot using Leap Motion. Okay, let's start with setting up a Leap Motion controller in Ubuntu and working with its ROS interface.