Book Description

A fast-paced, practical guide including examples, clear instructions, and details for building your own multimodal user interface

  • Step-by-step examples on how to master the essential features of Kinect technology
  • Fully-functioning code samples ready to expand and adjust to your need
  • Compact and handy reference on how to adopt a multimodal user interface in your application

In Detail

Kinect is a motion-sensing input device by Microsoft for the Xbox 360 video game console and Windows PCs. It provides capabilities to enhance human-machine interaction along with a zero-to-hero journey to engage the user in a multimodal interface dialog with your software solution.

Kinect in Motion - Audio and Visual Tracking by Example guides you in developing more than five models you can use to capture gestures, movements, and voice spoken commands. The examples and the theory discussed provide you with the knowledge to let the user become a part of your application.

Kinect in Motion - Audio and Visual Tracking by Example is a compact reference on how to master color, depth, skeleton, and audio data streams handled by Kinect for Windows.Starting with an introduction to Kinect and its characteristics, you will first be shown how to master the color data stream with no more than one page of lines of code. Learn how to manage the depth information and map them against the color ones. You will then learn how to define and manage gestures that enable the user to instruct the application simply by moving arms or any other type of natural action. Finally you will complete your journey through a multimodal interface, combining gestures with audio.The book will lead you through many detailed, real-world examples, and even guide you on how to test your application.

Table of Contents

  1. Kinect in Motion – Audio and Visual Tracking by Example
    1. Table of Contents
    2. Kinect in Motion – Audio and Visual Tracking by Example
    3. Credits
    4. About the Authors
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers and more
      2. Why Subscribe?
      3. Free Access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Downloading the color images of this book
        3. Errata
        4. Piracy
        5. Questions
    8. 1. Kinect for Windows – Hardware and SDK Overview
      1. Motion computing and Kinect
      2. Hardware overview
        1. The IR projector
        2. Depth camera
        3. The RGB camera
        4. Tilt motor and three-axis accelerometer
        5. Microphone array
        6. Software architecture
        7. Video stream
        8. Depth stream
        9. Audio stream
        10. Skeleton
      3. Summary
    9. 2. Starting with Image Streams
      1. Color stream
        1. Editing the colored image
        2. Image tuning
        3. The color image formats
        4. The Infrared color image format
        5. The raw Bayer formats
        6. YUV raw format
      2. Depth stream
        1. DepthRange – the default and near mode
        2. Extended range
        3. Mapping from the color frame to the depth frame
      3. Summary
    10. 3. Skeletal Tracking
      1. Tracking users
        1. Copying the skeleton data
      2. Default and Seated mode
      3. Detecting simple actions
        1. Joint rotations
      4. Summary
    11. 4. Speech Recognition
      1. Speech recognition
        1. Grammars
          1. A simple grammar sample
          2. The Microsoft.Speech library
      2. Tracking audio sources
        1. Sound source angle
        2. Beam angle
      3. Summary
    12. A. Kinect Studio and Audio Recording
      1. Kinect Studio – capturing Kinect data
      2. Audio stream data – recording and injecting
      3. Summary
    13. Index