Contents

About the Authors

About the Technical Reviewer

Acknowledgments

Introduction

Chapter 1: Getting Started

The Kinect Creation Story

Pre-History

The Minority Report

Microsoft's Secret Project

Microsoft Research

The Race to Hack Kinect

The Kinect for Windows SDK

Understanding the Hardware

Kinect for Windows SDK Hardware and Software Requirements

Step-By-Step Installation

Elements of a Kinect Visual Studio Project

The Kinect SDK Sample Applications

Kinect Explorer

Shape Game

Record Audio

Speech Sample

Summary

Chapter 2: Application Fundamentals

The Kinect Sensor

Discovering Connected a Sensor

Starting the Sensor

Stopping the Sensor

The Color Image Stream

Better Image Performance

Simple Image Manipulation

Taking a Snapshot

Reflecting on the objects

Data Retrieval: Events and Polling

Summary

Chapter 3: Depth Image Processing

Seeing Through the Eyes of the Kinect

Measuring Depth

Enhanced Depth Images

Better Shades of Gray

Color Depth

Simple Depth Image Processing

Histograms

Further Reading

Depth and Player Indexing

Taking Measure

Aligning Depth and Video Images

Depth Near Mode

Summary

Chapter 4: Skeleton Tracking

Seeking Skeletons

The Skeleton Object Model

SkeletonStream

SkeletonFrame

Skeleton

Joints

Kinect the Dots

The User Interface

Hand Tracking

Drawing the Puzzle

Solving the Puzzle

Expanding the Game

Space and Transforms

Space Transformations

Looking in the Mirror

SkeletonViewer User Control

Summary

Chapter 5: Advanced Skeleton Tracking

User Interaction

A Brief Understanding of the WPF Input System

Detecting User Interaction

Simon Says

Simon Says, “Design a User Interface”

Simon Says, “Build the Infrastructure”

Simon Says, “Add Game Play Infrastructure”

Starting a New Game

Enhancing Simon Says

Reflecting on Simon Says

Depth-Based User Interaction

Poses

Pose Detection

Reacting to Poses

Simon Says Revisited

Reflect and Refactor

Summary

Chapter 6: Gestures

Defining a Gesture

NUI

Where Do Gestures Come From?

Implementing Gestures

Algorithmic Detection

Neural Networks

Detection by Example

Detecting Common Gestures

The Wave

Basic Hand Tracking

Hover Button

Push Button

Magnet Button

Swipe

Magnetic Slide

Vertical Scroll

Universal Pause

The Future of Gestures

Summary

Chapter 7: Speech

Microphone Array Basics

MSR Kinect Audio

Speech Recognition

Audio Capture

Working with the Sound Stream

Cleaning Up the Sound

Canceling Acoustic Echo

Beam Tracking for a Directional Microphone

Speech Recognition

Summary

Chapter 8: Beyond the Basics

Image Manipulation Helper Methods

The Coding4Fun Kinect Toolkit

Your Own Extension Methods

Proximity Detection

Simple Proximity Detection

Proximity Detection with Depth Data

Refining Proximity Detection

Detecting Motion

Saving the Video

Identifying Faces

Holograms

Libraries to Keep an Eye On

Summary

Appendix: Kinect Math

Unit of Measure

Bit Manipulation

Bit Fields

Bitwise OR

Bitwise AND

Bitwise NOT (Complement)

Bit Shifting

Geometry and Trigonometry

Index