In this chapter, we will cover the following recipes:
- Getting a list of all the active users
- Identifying and coloring users' pixels in depth map
- Reading users' bounding boxes and center of mass
- Event-based reading of users' data
Until now we talked about OpenNI and learned how to use it for accessing devices and their raw outputs. But from this chapter onwards, we will cover NiTE, which is a middleware based on OpenNI and more focused on natural interactions. NiTE is a product of the same team as OpenNI and acts as middleware above the OpenNI framework with the purpose of providing more advanced outputs as a basic natural interaction interface for developers.
In this chapter, we will show you how to use the nite::UserTracker class to get the list of all active users in the scene and get their location and size in OpenNI's depth stream, along with the centre of visual mass of each user.
nite::NiTE, just as with openni::OpenNI, is a static class and a starting point when working with NiTE.
nite::NiTE contains three methods:
nite::NiTE::initialize(), nite::NiTE::shutdown(), and nite::NiTE::getVersion().
We are
not sure what nite::NiTE::initialize() actually does because NiTE is a closed source project; however, one of its jobs is to call
openni::OpenNI::initialize() so that developers don't need to initialize OpenNI and NiTE separately. We can also assume that nite::NiTE::shutdown()is used to free allocated memory and call openni::OpenNI::shutdown() at the end.
Except for the preceding methods, we don't have any use for the nite::NiTE object.
One of the important classes in NiTE is nite::UserTracker. This class is responsible for getting any information relative to users and their bodies. nite::UserTracker works like openni::VideoStream and has similar methods. However, it is actually not a sensor or a direct output of any physical unit. nite::UserTracker gives us what NiTE recognized from a depth stream of OpenNI. In this chapter, we are going to use only one of the methods of this class; more methods of this class will be covered in the next chapter:
nite::UserTrackerFrameRef is the object containing any available data about the current users. This class is used along with nite::UserTracker to give us a user-based view of a scene captured by the depth sensor. To get access to the latest produced nite::UserTrackerFrameRef object, you need to call the
nite::UserTracker::readFrame() method.
The nite::UserMap class is actually a bitmap data container or, in other words, a big array that has width x height x 2 bytes of data and is of the same size as the depth frame. But unlike the depth frame, there is no data about depth in these uint16 pixels of nite::UserMap. The values in nite::UserMap are the ID of a user to whom the pixel belongs, or the number zero if that pixel belongs to no user.
The nite::UserMap class has the following four methods:
nite::UserMap::getWidth(): The returned value is anintvalue showing the width ofnite::UserMap. This value is the same as the underlying depth frame's width.nite::UserMap::getHeight(): The returned value is anintvalue showing the height ofnite::UserMap. This value is the same as the underlying depth frame's height.nite::UserMap::getPixels(): This returns a pointer to the first pixel of data.nite::UserMap::getStride(): The returned value is in typeintand shows the number of bytes you need to add to the first pixel of a row to get the first pixel of the next row. Actually, it is the width of a row in bytes.
Each nite::UserTrackerFrameRef object contains a nite::UserMap object that can be retrieved using the nite::UserTrackerFrameRef::getUserMap() method.
nite::UserData is a class representing a user. In another words, nite::UserTrackerFrameRef gives you a nite::UserData object for each user it has recognized.
nite::UserData has a number of methods; we are going to use some in this chapter. These are as follows:
nite::UserData::getCenterOfMass(): This method returns anite::Point3fvalue that shows the center of visual mass (also known as COM and center of gravity) of a user. Anite::Point3fvalue is a structure of threefloatfields that are used to show a point in a 3D space.nite::UserData::getBoundingBox(): The return type in this method is thenite::BoundingBoxobject that contains twonite::Point3ffields, one for minimum coordinates and one for maximum coordinates. These two values can be used for calculating the 3D position and size of a user.nite::UserData::getId(): This method returns an integer showing a unique ID for this user. This ID is only guaranteed to be unique at the current time and may be used later again for someone else.nite::UserData::isLost(): This method returns abooltype value that can be used to indicate whether this user was not visible enough, that NiTE decided to drop him in the next frame of data.nite::UserData::isNew(): This method returns abooltype value that indicates whether this is the first appearance of a user.nite::UserData::isVisible(): This method returns abooltype value that gives us the visibility state of a user. A user can be invisible but yet not lost.
You can get access to a list (array) of all the nite::UserData objects by calling the nite::UserTrackerFrameRef::getUsers() method.