In this recipe we will learn how to track motion in the images produced from a webcam using OpenCV using the popular Lucas Kanade optical flow algorithm.
We will need to use OpenCV in this recipe, so please refer to the Integrating with OpenCV recipe from Chapter 3, Using Image Processing Techniques and add OpenCV and it's CinderBlock to your project. Include the following files to your source file:
#include "cinder/Capture.h" #include "cinder/gl/Texture.h" #include "CinderOpenCV.h"
Add the following using
statements:
using namespace ci; using namespace ci::app; using namespace std;
We will read frames from the camera and track motion.
ci::gl::Texture
and ci::Capture
objects to display and capture from a camera. Also, declare a cv::Mat
object as the previous frame, two std::vector<cv::Point2f>
objects to store the current and previous features, and a std::vector<uint8_t>
object to store the status of each feature:gl::Texture mTexture; Capture mCamera; cv::Mat mPreviousFrame; vector< cv::Point2f > mPreviousFeatures, mFeatures; vector< uint8_t > mFeatureStatuses;
setup
method we will initialize mCamera
:try{ mCamera = Capture( 640, 480 ); mCamera.start(); } catch( ... ){ console() << "unable to initialize device" << endl; }
update
method we need to check if mCamera
has been correctly initialized and whether it has a new frame available:if( mCamera ){ if( mCamera.checkNewFrame() ){
if
statements we will get a reference to ci::Surface
of mCamera
and then copy it to our mTexture
for drawing:Surface surface = mCamera.getSurface(); mTexture = gl::Texture( surface );
cv::Mat
with the current camera frame. We will also check if mPreviousFrame
contains any initialized data, calculate the good features to track, and calculate their motion from the previous camera frame to the current frame:cv::Mat frame( toOcv( Channel( surface ) ) ); if( mPreviousFrame.data != NULL ){ cv::goodFeaturesToTrack( frame, mFeatures, 300, 0.005f, 3.0f ); vector<float> errors; mPreviousFeatures = mFeatures; cv::calcOpticalFlowPyrLK( mPreviousFrame, frame, mPreviousFeatures, mFeatures, mFeatureStatuses, errors ); }
mPreviousFrame
and close the initial if
statements:mPreviousFrame = frame; } }
draw
method we will begin by clearing the background with black and drawing mTexture
:gl::clear( Color( 0, 0, 0 ) ); if( mTexture ){ gl::color( Color::white() ); gl::draw( mTexture, getWindowBounds() ); }
mFeatureStatus
to draw the features that have been matched:glColor4f( 1.0f, 0.0f, 0.0f, 1.0f ); for( int i=0; i<mFeatures.size(); i++ ){ if( (bool)mFeatureStatuses[i] == false ) continue; gl::drawSolidCircle( fromOcv( mFeatures[i] ), 5.0f ); }
mFeatureStatus
to draw one of the features that has been matched:for( int i=0; i<mFeatures.size(); i++ ){ if( (bool)mFeatureStatuses[i] == false ) continue; Vec2f pt1 = fromOcv( mFeatures[i] ); Vec2f pt2 = fromOcv( mPreviousFeatures[i] ); gl::drawLine( pt1, pt2 ); }
In the following image, the red dots represent good features to track:
The optical flow algorithm will make an estimation of how much the tracked point has moved from one frame to the other.
In this recipe we are using the cv::goodFeaturesToTrack
object to calculate which features are optimal for tracking, but it is also possible to manually choose which points we wish to track. All we have to do is populate mFeatures
manually with whatever points we wish to track and pass it to the cv::calcOpticalFlowPyrLK
. object
18.117.11.247