So, what are “interactive displays”? We define them to be the displays that not only show visual information on the screens, but also sense and understand human actions and receive direct user inputs. Interactive displays that can “feel” the touch of our fingers are already ubiquitous, especially on mobile devices and all-in-one computers. Now, the addition of human-like perceptual sensing and recognition technologies is allowing the development of a new class of interactive displays and systems that can also “see”, “hear”, and “understand” our actions in the three-dimensional space in front of and around them.

We use multisensory and multimodal interface schemes to comprehend the physical world surrounding us and to communicate with each other in our daily lives, seamlessly combining multiple interaction modalities such as touch, voice, gestures, facial expressions, and eye gaze. If we want human-device interactions to approach the richness of human-human interactions, then we must endow the devices with technologies to sense and understand such natural user inputs and activities. The addition of natural human-interfaces can thus bring lifelike experiences to human-device interactions.

The ways in which we interact with computers have already gone through a transformation in recent decades, with graphical user interfaces that use a mouse and keyboard as input devices replacing the old command-line interfaces that used text-based inputs. We are now witnessing the next revolution, with the advent of natural human interfaces where the user interacts with computing devices using touch, gesture, voice, etc. The ultimate goal of implementing a human-device interface scheme is to make the interaction experiences natural, intuitive and immersive for the user. While the limitations of the technologies at hand require designers and engineers to make compromises and to settle for a subset of these goals for specific product implementations, significant advances have been taking place in recent years towards realizing this objective.

This book presents an in-depth review of the technologies, applications, and trends in the rapidly emerging field of interactive displays, focusing on natural human interfaces. The first chapter of the book starts with a review of the basics of human sensing and perception processes, and an overview of human-device interactions utilizing natural interface technologies based on the sensing and inference of touch, voice, and vision. The subsequent chapters delve into the details of each of these input and interaction modalities, providing in-depth discussion on the fundamentals of the technologies and their applications in human interface schemes, as well as combinations of them, towards realizing multisensory and multimodal interactions. The book concludes with a chapter on the fundamental requirements, technology development status, and outlook towards realizing “true” 3D interactive displays that would provide lifelike immersive interaction experiences.

I would like to thank the series editor, Anthony Lowe, for recognizing the need for a book on interactive displays as part of the Wiley-SID Series on Display Technology. I am grateful to the experts from across the industry and academia who contributed to this book. I also appreciate the support of the production staff at John Wiley.

Finally, I dedicate this book to Shida, Rohan, and Ava, without whose encouragement and support I would not have been able to undertake and complete this project.

Achintya K. Bhowmik
Cupertino, California, USA
February 2014