To capture sensor data directly in a binary format, implement the following function:

void writeDepthBuffer(openni::VideoFrameRef depthFrame){
  static int depth_buffer_counter=0;
  char file_name [512];
  sprintf(file_name, "%s%d.bin", "depth_frame", depth_buffer_counter);
  openni::DepthPixel *depthPixels = new openni::DepthPixel[depthFrame.getHeight()*depthFrame.getWidth()];
  memcpy(depthPixels, depthFrame.getData(), depthFrame.getHeight()*depthFrame.getWidth()*sizeof(uint16_t));
  std::fstream myFile (file_name, std::ios::out |std::ios::binary);
  myFile.write ((char*)depthPixels, depthFrame.getHeight()*depthFrame.getWidth()*sizeof(uint16_t));
  depth_buffer_counter++;
  printf("Dumped Depth Buffer %d
",depth_buffer_counter);
  myFile.close();
  delete depthPixels;
}

Similarly, we also capture the raw RGB color data with the following implementation:

  void writeColorBuffer(openni::VideoFrameRef colorFrame){
    static int color_buffer_counter=0;
    char file_name [512];
    sprintf(file_name, "%s%d.bin", "color_frame", color_buffer_counter);
    //basically unsigned char*
    const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)colorFrame.getData();
    std::fstream myFile (file_name, std::ios::out | std::ios::binary);
    myFile.write ((char*)imageBuffer, colorFrame.getHeight()*colorFrame.getWidth()*sizeof(uint8_t)*3);
    color_buffer_counter++;
    printf("Dumped Color Buffer %d, %d, %d
", colorFrame.getHeight(), colorFrame.getWidth(), color_buffer_counter);
    myFile.close();
  }

The preceding code snippet can be added to any sample code within the OpenNI2 SDK that provides depth and color data visualization (to enable raw data capture). We recommend that you modify the Viewer.cpp file in the OpenNI2-master/Samples/SimpleViewer folder. The modified sample code is included in our code package. To capture raw data, press R and the data will be stored in the depth_frame0.bin and color_frame0.bin files.

The depth sensor returns two streams of data in real time. One data stream is a 3D depth map, which is stored in 16-bits unsigned short data type (see the following figure on the left-hand side). Another data stream is a color image (see the following figure on the right-hand side), which is stored in a 24 bits per pixel, RGB888 format (that is, the memory is aligned in the R, G, and B order, and 8 bits * 3 channels = 24 bits are used per pixel).

The binary data is written directly to the hard disk without compression or modification to the data format. On the client side, we read the binary files as if there is a continuous stream of data and color data pairs arriving synchronously from the hardware device. The OpenNI2 driver provides the mechanism to interface with the PrimeSense-based sensors (Microsoft Kinect or PS1080).

The openni::VideoFrameRef depthFrame variable, for example, stores the reference to the depth data buffer. By calling the depthFrame.getData() function, we obtain a pointer to the buffer in the DepthPixel format, which is equivalent to the unsigned short data type. Then, we write the binary data to a file using the write() function in the fstream library. Similarly, we perform the same task with the color image, but the data is stored in the RGB888 format.

Additionally, we can enable the setImageRegistrationMode (openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR) depth map registration function in OpenNI2 to automatically compute and map a depth value onto the color image. The depth map is overlaid onto the color image and is shown in the following figure:

In the next section, we will assume that the raw depth map is precalibrated with image registration by OpenNI2 and can be used to compute the real-world coordinates and UV mapping indices directly.