Index

A

Ambient light, 12

B

Blurring algorithm, 68

Brightness thresholding algorithm

float testApp::blur, 70

kinect.update(), 69

void testApp::draw(), 70

void testApp::exit(), 70

void testApp::setup(), 69

void testApp::update(), 69

C, D, E, F

Computer vision

image anatomy, 65

image comparison, 74

background subtraction, 76–80

black and white image creation, 85–86

double image, 85

frame differencing, 80–84

image storage, 87

tolerance, 75–76

image processing. see Image processing

G

Gesture recognition, 89

definiton, 89

multitouch detection

assigning and tracking component IDs, 95–96

camera image, background storing and subtracting, 92

connected components algorithm, 93–95

fingertip touching, 90

image processing, 90–91

infrared emitter and detector, 89

Kinect's depth image, 101

LCD display, 89

minority report—style interface, 99–100

motion, 97

multitouch-capable devices, 96

rotation, 97–99

scale, 99

shape, 101

threshold filter, 90, 92–93

H

Hardware

accelerometer, 23–24

depth sensing, 11–13

RGB camera. see RGB camera

tilting head, 23–24

volumetric sensing

Arduino Sketch, 35, 36, 38

binary distributions, 26

block wiring, 37

//BufferedAsync Setup, 32, 34

CMakeLists.txt file, 34–35

///Keyboard Event Tracking, 29, 30

///Kinect hardware connection class, 27

lit alarm light, 39

///Mutex Class, 26, 27

//~MyFreenectDevice(), 27

parts, 25

//PCL, 29

//Percentage Change, 30, 32

relay wiring, 35, 36

///Start the PCL/OK Bridging, 28–29

I, J

Image processing

brightest pixel tracking, 72–74

brightness thresholding algorithm. see Brightness thresholding algorithm

Image processing (cont.)

data simplifying, 66–67

noise and blurring, 67–68

situation contriving, 69

Infrared emitter and detector, 89

K

Kinect

drivers installation:. see OpenKinect driver

hardware requirements, 1–2

installation testing, 9

L

Linux, 6–7

M, N

Mac OS X, 8–9

Mesh Models:, 128

Multiple kinects

calibration, 209

calib.yaml file, 229

camera frame, 228

eigen.hpp, 243

OKStereo.cpp, 229–243

stereo calibration, 228

world frame, 228

depth shadows, occlusions, 208

field of view, 207

hardware requirements, 209

interference, 209

angle and distance, 213, 214

box fan test, 217

calib.yaml file, 226

cloud 1 update, 227

cloud 2 update, 227

combined point clouds, 226

hardware shutter system, 218

holes, 209, 212

IR camera, 218

IR pattern, 209–211

laser diodes, 217

mechanical shutters, 217

noise, 209

OKShutter.cpp, 219–226

scene, 213

splotches, 209, 213

single direction, 207

Multitouch detection, gesture recognition

assigning and tracking component IDs, 95–96

camera image, background storing and subtracting, 92

connected components algorithm, 93–95

fingertip touching, 90

image processing, 90–91

infrared emitter and detector, 89

Kinect's depth image, 101

LCD display, 89

minority report—style interface, 99–100

motion, 97

multitouch-capable devices, 96

rotation, 97–99

scale, 99

shape, 101

threshold filter, 90, 92–93

O

Object detection

global descriptors

CloudRecognizer Class, 201, 202

database model, 202

VFH descriptor computation, 202–204

pose estimation, 204–205

Object modeling

3-D

camera space, 191

cleaning and cropping, partial views, 195–196

high-resolution models, 199

Kinect pose estimation, 192–195

marker-based scanner, 189–190

Point clouds merging, 196–199

support builiding, 191

definition, 173

single Kinect image. see Single Kinect image

OpenGL

drawing points, 135

initialization code, 133–135

OpenKinect driver

Linux, 6–7

Mac OS X, 8–9

Red Hat/Fedora, 7

Ubuntu, 7

Windows, 2

CMake preconfiguration, 5

Git Commands, 3

libfreenect, 3

Microsoft Visual Studio 2010 and MinGW, 5–6

updation, 3

P, Q

Person tracking, 123–125

Point cloud library (PCL), 129

OpenKinect

binary distributions, 57

// Create and setup the viewer, 60

C++ file creation, 56

—CMakeLists.txt—, 62

///Kinect Hardware Connection Class, 58

///Mutex Class, 58

//~MyFreenectDevice(), 58

//More Kinect Setup, 60

///Start the PCL/OK Bridging, 59

Windows installation

cmake-guifor FLANN, 49

CMinPack, 49

Linux, 52–53

Mac OS X, 53–55

Qhull, 51

VTK installer, 51

Point clouds

coloring

depth to color reference frame, 131

image plane, 132

Depth Map, 130–131

2-D registration

affine transformation, 154

matched features, 152

transformation parameters, 153

translation, 153, 154

3-D data representation

Mesh Models, 128

rendering, 129

scaling pixel count, 127, 128

Voxels, 128

3-D registration

absolute orientation, 156–157

rigid transformation, 155

rotation computation, 155

outliers, 157–159

PCL creation, 139–142

SLAM. See Simultaneous Localization and Mapping

surface reconstruction

normal estimation, 162

triangulation method, 162–163

visualization

with OpenGL, 133–135

with PCL, 133

wind application

blue-red gradient, 136

Freenect Thread Code, 137–139

intensity field, 142

is_frozen, 142

Kinect depth image, 139–142

libraries, 136–137

main() function, 139

OpenGL Code, 143–149

radiohead's video, 136

screenshot, 149

show_visualizer(), 142

structure of, 136

TMyPoint, 142

R

Random sample consensus (RANSAC), 174

Red Hat/Fedora, 7

RGB camera

build/bin/calibrate_kinect_ir execution, 18

calibration target, 13, 16–17

Capture directory, 17–18

Combined R|T matrix, 23

kinect_calibration.yml file, 19–21

Linux, 15

Mac OS X, 16

output image, 18, 19

pinhole model, 21, 22

rgb_distortion and depth_distortion, 21, 22

rgb_intrinsics/depth_intrinsics, 21

rgbd-viewer, 17

Windows installation, 14–15

S

Shape gestures, 101

Simultaneous localization and mapping (SLAM)

advantages of, 160

conventional camera, 159

Kinect, 160–161

real-time considerations, 161

simple Kinect

C++ classes, 164

camera pose estimation, 170–172

CTrackingSharedData class, 166

SLAM, simple kinect (cont.)

main classes of, 164–165

median feature computation, 168

Point Map construction, 169–170

screenshot, 164

SURF, 167–168

Single Kinect image

3-D Model

extruder class, 180

Mesh building, 187–188

surface point cloud, 181, 185–187

unseen Voxels, 183–185

Voxelized representation, 181–183

parametric model, 178–179

tabletop object detector

background removal, 176

individual object clusters extraction, 177–178

points lying, prism, 177

sample scene, 174

table plane extraction, 174–176

Software

Kinect drivers

Microsoft Kinect SDK, 41

OpenKinect, 41

OpenNI, 41

OpenCV installation

Linux, 52–53

Mac OS X, 53–55

Windows, 42–43

point cloud library (PCL) installation

// Create and setup the viewer, 60

///Kinect Hardware Connection Class, 58

///Mutex Class, 58

///Start the PCL/OK Bridging, 59

//~MyFreenectDevice(), 58

//More Kinect Setup, 60

binary distributions, 57

C++ file creation, 56

—CMakeLists.txt—, 62

Structured light pattern, 12

T

Tabletop object detector

background removal, 176

individual object clusters extraction, 177–178

points lying, prism, 177

sample scene, 174

table plane extraction, 174–176

Threshold filter, 92–93

U

Ubuntu, 7

V

Volumetric sensing

OKFlower.cpp

Arduino Sketch, 35, 36, 38

binary distributions, 26

block wiring, 37

//BufferedAsync Setup, 32, 34

CMakeLists.txt file, 34–35

///Keyboard Event Tracking, 29, 30

///Kinect hardware connection class, 27

lit alarm light, 39

///Mutex Class, 26, 27

//~MyFreenectDevice(), 27

//PCL, 29

//Percentage Change, 30, 32

relay wiring, 35, 36

///Start the PCL/OK Bridging, 28–29

parts, 25

Voxelization, 103

clustering voxels, 122

cluster_indices, 122

2-D flood fill technique, 120

EuclideanClusterExtraction, 121

KdTree line, 122

PCL, 121–122

setClusterTolerance, 122

setMinClusterSize and setMaxClusterSize, 122

dataset, 104

definition, 103–104

manipulating voxels

background cloud, 118

background subtraction, 108–116

drawing voxel boxes, 108

foreground cloud, 117, 120

full scene cloud, 117, 119

function, background subtraction, 116–117

getPointIndicesFromNewVoxels, 117

leaf nodes, 107

octrees, 105–107

PCL, 105

tracking people and fitting rectangular prism, 122–125

Voxels, 128

W, X, Y, Z

Wind application

animation code, 142–143

blue-red gradient, 136

Freenect Thread Code, 137–139

intensity field, 142

is_frozen, 142

Kinect depth image, 139–142

libraries, 136–137

main() function, 139

OpenGL Code, 143–149

screenshot, 149

show_visualizer(), 142

structure of, 136

TMyPoint, 142