Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Object tracking

In this recipe, we will learn how to track specific planar objects in our webcam using OpenCV and it's corresponding CinderBlock.

Getting ready

You will need an image depiction of the physical object you wish to track in the camera. For this recipe place that image in the assets folder and name it object.jpg.

We will use the OpenCV CinderBlock in this recipe, so please refer to the Integrating with OpenCV recipe from Chapter 3, Using Image Processing Techniques and add OpenCV and it's CinderBlock to your project.

If you are using a Mac, you will need to compile the OpenCV static libraries yourself, because the OpenCV CinderBlock is missing some needed libraries on OSX (it will work fine on Windows). You can download the correct version from the following link: http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.3/.

You will need to compile the static libraries yourself using the provided CMake files. Once your libraries are correctly added to your project, include the following files:

#include "cinder/Capture.h"
#include "cinder/gl/Texture.h"
#include "cinder/ImageIo.h"

Add the following using statements:

using namespace ci;
using namespace ci::app;
using namespace std;

How to do it…

We will track an object in the camera frames based on an image depicting the object

Let's begin by creating a struct method to store the necessary objects for feature tracking and matching. Add the following code before your application class declaration:
```
struct DetectionInfo{
    vector<cv::Point2f> goodPoints;
    vector<cv::KeyPoint> keyPoints;
    cv::Mat image, descriptor;
    gl::Texture texture;
};
```

In your class declaration add the following member objects:

DetectionInfo mObjectInfo, mCameraInfo;
    cv::Mat mHomography;
    cv::SurfFeatureDetector mFeatureDetector;
    cv::SurfDescriptorExtractor mDescriptorExtractor;
    cv::FlannBasedMatcher mMatcher;
    vector<cv::Point2f> mCorners;

In the setup method let's start by initializing the camera:

    try{
        mCamera = Capture( 640, 480 );
        mCamera.start();
    } catch( ... ){
        console() << "could not initialize capture" << endl;
    }

Lets resize mCorners, load our object image, and calculate its image, keyPoints, texture, and descriptor:

mCorners.resize( 4 );
    Surface objectSurface = loadImage( loadAsset( "object.jpg" ) );
    mObjectInfo.texture = gl::Texture( objectSurface );
    mObjectInfo.image = toOcv( Channel( objectSurface ) );
    mFeatureDetector.detect( mObjectInfo.image, mObjectInfo.keyPoints );
    mDescriptorExtractor.compute( mObjectInfo.image, mObjectInfo.keyPoints, mObjectInfo.descriptor );

In the update method, we will check if mCamera has been initialized and whether we have a new frame to process:
```
    if( mCamera ){
        if( mCamera.checkNewFrame() ){
```
Now let's get the surface of mCamera and initialize texture and image objects of mCameraInfo. We will create a ci::Channel object from cameraSurface that converts color surfaces to gray channel surfaces:
```
Surface cameraSurface = mCamera.getSurface();
mCameraInfo.texture = gl::Texture( cameraSurface );
mCameraInfo.image = toOcv( Channel( cameraSurface ) );
```

Let's calculate features and descriptor values of mCameraInfo:

mFeatureDetector.detect( mCameraInfo.image, mCameraInfo.keyPoints);
mDescriptorExtractor.compute( mCameraInfo.image, mCameraInfo.keyPoints, mCameraInfo.descriptor );

Now let's use mMatcher to calculate the matches between mObjectInfo and mCameraInfo:

            vector<cv::DMatch> matches;
            mMatcher.match( mObjectInfo.descriptor, mCameraInfo.descriptor, matches );

To perform a test to check for false matches, we will calculate the minimum distance between matches:

double minDist = 640.0;
for( int i=0; i<mObjectInfo.descriptor.rows; i++ ){
                double dist = matches[i].distance;
                if( dist < minDist ){
                    minDist = dist;
                }
            }

Now we will add all the points whose distance is less than minDist*3.0 to mObjectInfo.goodPoints.clear();

mCameraInfo.goodPoints.clear();
for( vector<cv::DMatch>::iterator it = matches.begin(); 
 it != matches.end(); ++it ){
 if( it->distance < minDist*3.0 ){
  mObjectInfo.goodPoints.push_back( 
   mObjectInfo.keyPoints[ it->queryIdx ].pt );
  mCameraInfo.goodPoints.push_back( 
   mCameraInfo.keyPoints[ it->trainIdx ].pt );
 }

} With all our points calculated and matched, we need to calculate the homography between the points of mObjectInfo and mCameraInfo:
```
mHomography = cv::findHomography( mObjectInfo.goodPoints, mCameraInfo.goodPoints, CV_RANSAC );
```

Let's create vector<cv::Point2f> with the corners of our object and perform a perspective transform to calculate the corners of our object in the camera image:

Tip

Don't forget to close the brackets we opened earlier.

  vector<cv::Point2f> objCorners( 4 );
  objCorners[0] = cvPoint( 0.0f, 0.0f );
  objCorners[1] = cvPoint( mObjectInfo.image.cols, 0.0f);
  objCorners[2] = cvPoint( mObjectInfo.image.cols, 
   mObjectInfo.image.rows );
  objCorners[3] = cvPoint( 0.0f, mObjectInfo.image.rows);
  mCorners = vector< cv::Point2f >( 4 );
  cv::perspectiveTransform( objCorners, mCorners, 
   mHomography );
 } 
}

Let's move to the draw method and begin by clearing the background and drawing the camera and object textures:

  gl::clear( Color( 0, 0, 0 ) );

    gl::color( Color::white() );
    if( mCameraInfo.texture ){
        gl::draw( mCameraInfo.texture, getWindowBounds() );
    }

    if( mObjectInfo.texture ){
        gl::draw( mObjectInfo.texture );
    }

Now let's iterate over goodPoints values in both mObjectInfo and mCameraInfo and draw them:

for( int i=0; i<mObjectInfo.goodPoints.size(); i++ ){
 gl::drawStrokedCircle( fromOcv( mObjectInfo.goodPoints[ i ] ),
  5.0f );
 gl::drawStrokedCircle( fromOcv( mCameraInfo.goodPoints[ i ] ),
  5.0f );
 gl::drawLine( fromOcv( mObjectInfo.goodPoints[ i ] ), 
  fromOcv( mCameraInfo.goodPoints[ i ] ) );
}

Now let's iterate over mCorners and draw the corners of the found object:

gl::color( Color( 1.0f, 0.0f, 0.0f ) );
    gl::begin( GL_LINE_LOOP );
    for( vector<cv::Point2f>::iterator it = mCorners.begin(); it != mCorners.end(); ++it ){
        gl::vertex( it->x, it->y );
    }
    gl::end();

Build and run the application. Grab the physical object you depicted in the object.jpg image and put it in front of the image. The program will try to track that object in the camera image and draw it's corners in the image.

How it works…

We are using a Speeded Up Robust Features (SURF) feature detector and descriptor to identify features. In the step 4, we are calculating the features and descriptor. We use a cv::SurfFeatureDetect object or that calculates good features to track on our object. The cv::SurfDescriptorExtractor object then uses these features to create a description of our object. In the step 7, we do the same for the camera image.

In the step 8, we then use a Fast Library for Approximate Nearest Neighbor (FLANN) called cv::FlannBasedMatcher. This matcher takes the description from both the camera frame and our object, and calculates matches between them.

In steps 9 and 10, we use the minimum distance between matches to eliminate the possible false matches. The result is passed into mObjectInfo.goodPoints and mCameraInfo.goodPoints.

In the step 11, we calculate the homography between image and camera. A homography is a projection transformation from one space to another using projective geometry. We use it in the step 12 to apply a perspective transformation to mCorners to identify the object corners in the camera image.

There's more…

To learn more about what SURF is and how it works, please refer to the following web page: http://en.wikipedia.org/wiki/SURF.

To learn more about FLANN, please refer to the web page http://en.wikipedia.org/wiki/Nearest_neighbor_search.

To learn more about homography please refer to the following web page:

http://en.wikipedia.org/wiki/Homography.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Object tracking

Create new playlist

Sign In

Sign Up

Object tracking

Getting ready

How to do it…

Tip

How it works…

There's more…

Table of Contents for
Object tracking