OpenNI is a multilanguage, cross-platform framework that defines APIs in order to write applications using natural interaction (NI) (see https://structure.io/openni for more information). Natural interaction refers to the way in which people naturally communicate through gestures, expressions, and movements, and discover the world by looking around and manipulating physical objects and materials.

OpenNI APIs are composed of a set of interfaces that are used to write NI applications. The following figure shows a three-layered view of the OpenNI library:

The top layer represents the application layer that implements the natural interaction-based application. The middle layer is the OpenNI layer, and it will provide communication interfaces that interact with sensors and middleware components that analyze the data from the sensor. Middleware can be used for full-body analysis, hand-point analysis, gesture detection, and so on. One example of a middle layer component is NITE (http://www.openni.ru/files/nite/index.html), which can detect gestures and skeletons.

The bottom layer contains the hardware devices that capture the visual and audio elements of the scene. It can include 3D sensors, RGB cameras, IR cameras, and microphones.

The latest version of OpenNI is OpenNI 2, which support sensors such as Asus Xtion Pro, and Primesense Carmine. The first version of OpenNI mainly supports the Kinect 360 sensor.

OpenNI is cross platform, and has been successfully compiled and deployed on Linux, Mac OS X, and Windows.

In the next section, we will see how we to install OpenNI in Ubuntu.