Starting the device

Now it's time to initialize the Kinect device in our own application. There will be a lot of Kinect API functions for us to use without any preparatory lessons. But don't worry; you will find that most of them are self-explanatory and easy to understand. Also, we will introduce each function and their parameters in the Understanding the code section.

We will continue working on the framework we have just created, so existing code lines will not be listed here again.

Initializing and using Kinect in C++

Now we can try to find and start the Kinect device in our own C++ framework.

  1. Add the following include files:
    #include <MSHTML.h>
#include <NuiApi.h>
#include <sstream>
  2. Add the necessary global variables for use in all functions:
    INuiSensor* context = NULL;
HANDLE colorStreamHandle = NULL;
HANDLE depthStreamHandle = NULL;
std::string hudText;
  3. Add an initializeKinect() function, which will be called before the GLUT main loop. It returns false if the process fails for any reason.
    // Check if there are any Kinect sensors connected with
// current PC and obtain the number
int numKinects = 0;
HRESULT hr = NuiGetSensorCount( &numKinects );
if ( FAILED(hr) || numKinects<=0 ) return false;

// Create the sensor object and set it to context
.
// Here we only use the first device (index 0) we find.
hr = NuiCreateSensorByIndex( 0, &context );
if ( FAILED(hr) ) return false;

// Initialize the sensor with color/depth/skeleton enabled
DWORD nuiFlags = NUI_INITIALIZE_FLAG_USES_SKELETON |
                 NUI_INITIALIZE_FLAG_USES_COLOR |
                 NUI_INITIALIZE_FLAG_USES_DEPTH;
hr = context->NuiInitialize( nuiFlags );
if ( FAILED(hr) ) return false;

// Open color and depth video streams for capturing.
// The resolution is set to 640x480 here.
hr = context->NuiImageStreamOpen(
    NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480,
    0, 2, NULL, &colorStreamHandle );
if ( FAILED(hr) ) return false;

hr = context->NuiImageStreamOpen(
    NUI_IMAGE_TYPE_DEPTH, NUI_IMAGE_RESOLUTION_640x480,
    0, 2, NULL, &depthStreamHandle );
if ( FAILED(hr) ) return false;

// Enable skeleton tracking
hr = context->NuiSkeletonTrackingEnable( NULL, 0 );
if ( FAILED(hr) ) return false;
return true;
  4. Add a destroyKinect() function after the main loop in which we just release the sensor object we created before.
    if ( context )
    context->NuiShutdown();
  5. In the main entry, we alter the last few lines as follows:
    if ( !initializeKinect() ) return 1;
glutMainLoop();
destroyKinect();
return 0;
  6. The program can compile and run now. But it still produces nothing. We don't know whether Kinect works or not as it shows a blank window. So, next we will add a few lines in update() and render() to print some continuous updated Kinect information.
  7. At the beginning of update(), obtain one color frame and output the current frame number and time stamp value into a string:
    NUI_IMAGE_FRAME colorFrame;
HRESULT hr = context->NuiImageStreamGetNextFrame(
    colorStreamHandle, 0, &colorFrame );
if ( SUCCEEDED(hr) )
{
    std::stringstream ss;
    ss << "Frame: " << colorFrame.dwFrameNumber << "  "
       << "Time: " << (double)colorFrame.liTimeStamp.QuadPart * 0.001;
    hudText = ss.str();
    context->NuiImageStreamReleaseFrame(
        colorStreamHandle, &colorFrame );
}
  8. In the render() function, render the text on screen as follows:
    // Clear last frame buffer
glClearColor( 0.0f, 0.0f, 0.0f, 0.0f );
glClear( GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT );

// Set up the projection matrix for text display
glMatrixMode( GL_PROJECTION );
glLoadIdentity();
glOrtho( 0.0, 1.0, 0.0, 1.0, -1.0, 1.0 );

glMatrixMode( GL_MODELVIEW );
glLoadIdentity();

// Print the text at the bottom of the window
glRasterPos2f( 0.01f, 0.01f );
glColor4f( 1.0f, 1.0f, 1.0f, 1.0f );
glutBitmapString( GLUT_BITMAP_TIMES_ROMAN_24,
                  (const unsigned char*)hudText.c_str() );
  9. Now, execute the compiled program; you may find it a little slower when you start executing. Be patient until the depth sensor starts to lighten. The screen is still dark but you will find a line of animation text at the bottom-left as shown in the following figure:
    Initializing and using Kinect in C++

    A snapshot of the application

  10. If the application directly exits without displaying anything, the initialization process may fail. Add some text before returning false to see the value of hr in initializeKinect(); also check if your Kinect sensor is connected and not used by other programs.

Understanding the code

The following table shows all the Kinect functions we have used as well as descriptions of the important parameters.

Function/method name

Parameters

Description

NuiGetSensorCount

int* pCount

Get the number of Kinect sensors connected to the PC and set to pCount.

NuiCreateSensorByIndex

int index, INuiSensor** ppNuiSensor

Creates an instance of the Kinect sensor at index and sets it to ppNuiSensor.

INuiSensor::NuiInitialize

DWORD dwFlags

Initialize Kinect with specified feature options, including audio, color, depth, depth with player index, and skeleton.

INuiSensor::NuiImageStreamOpen

NUI_IMAGE_TYPE type,

NUI_IMAGE_RESOLUTION res,

DWORD dwFrameFlags,

DWORD dwFrameLimit,

HANDLE hNextFrameEvent,

HANDLE *phStreamHandle

  • Open an image stream with specific type (color, depth, and so on) and resolution (640x480 for our case), and set its handle to phStreamHandle.
  • The dwFrameFlags provides some additional options (default is 0).
  • The dwFrameLimit means frame numbers limited for buffering (always set to 2).
  • The hNextFrameEvent is used for multithreaded cases.

INuiSensor::NuiSkeletonTrackingEnable

HANDLE hNextFrameEvent,

DWORD dwFlags

Enable skeleton tracking with additional options dwFlags (default is 0).

The hNextFrameEvent is used for multithreaded cases.

INuiSensor::NuiShutdown

 

Turns Kinect off.

Note

All functions and methods start with the prefix "Nui". It is just short for Natural User Interface(NUI).

Now the total process of creating and using Kinect in user applications can be summarized as follows:

  1. Find and create the sensor object.
  2. Initialize the sensor with the required features (image streams and skeleton tracking), and enable these features.
  3. Update every frame to get stream and skeleton data for use.
  4. Release the sensor object when exiting.

Quite simple, isn't it? Note that, we didn't introduce the lines in the update() function here. We will explain that in the next chapter, with more interesting live images shown on the screen instead of a boring line of text.

Additional information

Another interesting and challenging task is to implement a multithreaded version of the initialization and updating of Kinect. In fact, some functions here have already supported such uses by accepting event handles as parameters, including NuiImageStreamOpen() and NuiSkeletonTrackingEnable(). Events will change when new video stream/skeleton frames arrive, so we can listen to them with WaitForMultipleObjects() in a separate thread and then obtain related frame data.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.252.140