For the past several years, I've been learning to play the cello. As any adult who has ever tried to learn a musical instrument—especially a fretless stringed instrument—can tell you, it's painfully hard. You feel like you don't really know your body at all: your muscles get sore in odd places, you develop strange calluses on your fingers, and you perform stretches with your hands you never thought possible. It requires almost all of your concentration simply to coax the instrument into making a pleasing sound, which is very different from actually making music.
But, ah, when you do begin to make music, it is a thing of joy. When you are able to get a sweet sound with emotional resonance from an inanimate object using only your body, it is an amazing feeling. Gestures create meaning.
That's what this book is about.
Technological, social, and market forces have converged to create a fertile new ground for designers and engineers to plow. The price of processing speed has dropped and sensors are readily available. Touchscreens on our mobile devices, ATMs, and airline check-in kiosks have taught us to expect to be able to manipulate things on-screen with our hands. Games have shown us we can make gestures in space to control objects on-screen. Public restrooms are, believe it or not, test laboratories for interactive gestures: placing your hands under a faucet to turn it on, waving your hands to get a paper towel, stepping into a room to turn on the lights.
All of these things have ushered in a new era of interaction design, one where gestures on a surface and in the air replace (or at least supplement) keyboards, mice, and styli. This new era, however, means those who design and develop more "traditional" systems need to grow their skills, adding in knowledge about kinesiology, sensors, ergonomics, physical computing, touchscreen technology, and new interface patterns.
That's what this book is about.
I wrote this book because, on my third gestural interface project, I was frustrated that I could find only bits and pieces of information about a subject—this subject—that was obviously important and growing rapidly. Why wasn't there a book about these things? I groused. How come I have no idea how big a touchscreen button needs to be? What is a capacitive touchscreen? What kinds of gestures can I use? And thus the idea for the book you are reading was born.
Aside from the constraints of the human body, many of the other parts of this book were changing even as I wrote about them, and new products that utilized interactive gestures seemingly came out every week, if not several times a week, over the course of the nine months I wrote this book (November 2007–July 2008). The newness of the subject was glaringly apparent, but I have tried, when and where possible, to emphasize what is unlikely to change and techniques that are, if not proven, at least being used by practicing designers and developers in the field today.
This book is mostly for interaction and industrial designers who have found themselves in the same position I found myself in several years ago: being asked to design gestural interfaces (especially touchscreens) and, coming from the world of web and desktop software or physical products, having no central place to go to for information to get started. I hope this book is a starting place and a reference point.
And even though there isn't a line of code in this book, I also wrote it with developers in mind, knowing full well that they are often the people who have to take what designers dream up and make it real. I hope everyone involved in creating new products that make use of sensors and interactive surfaces will find something of use in these pages.
This book is divided into roughly four parts. Although certainly you can read the book straight through, you don't have to. I recommend that everyone start with Chapter 1 and Chapter 2 just to provide a background for what comes later.
The next section, comprising Chapter 3 and Chapter 4, is meant to be used as reference material, mostly when you're designing or are in the process of designing. The patterns show how other designers have solved interface challenges in the past.
Chapter 5 through Chapter 7 are about the process of design, from documentation to prototyping to communicating what the product does to its audience. You also can use these chapters for reference as necessary.
Chapter 8 takes a look at future trends in this subject—a future that gets closer and closer every day. It was, after all, only six years ago that the gestural interfaces in Minority Report were science fiction, and now we can see them being deployed everywhere. "The future is here," as William Gibson famously noted. "It's just not evenly distributed."
Designers can flip through the appendix for inspiration, especially when creating free-form gestural interfaces.
Please address comments and questions concerning this book to the publisher:
| O'Reilly Media, Inc. |
| 1005 Gravenstein Highway North |
| Sebastopol, CA 95472 |
| 800-998-9938 (in the U.S. or Canada) |
| 707-829-0515 (international/local) |
| 707-829-0104 (fax) |
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at:
| http://www.oreilly.com/catalog/9780596518394 |
To comment or ask technical questions about this book, send email to:
| [email protected] |
For more information about our books, conferences, Resource Centers, and the O'Reilly Network, see our website at:
| http://www.oreilly.com |
When you see a Safari® Books Online icon on the cover of your favorite technology book, that means the book is available online through the O'Reilly Network Safari Bookshelf.
Safari offers a solution that's better than e-books. It's a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com.
It's easy to come up with an idea for a book, but it's much harder to write one. It's a task that, like most other products, requires lots of people to make it a reality.
This book would not be possible without the patience, support, and 6:00 a.m. writing wake-up calls of my wife, Rachael King. Her encouragement and understanding kept me going on those days when I could eek out only a sentence—which were many.
My daughter Fiona's insistence that I not work on this book and instead play Wii with her also, I'm sure, kept me sane and contributed vastly to the quality of this tome. (At the very least, I got to practice many interactive gestures!)
My family and friends have my undying gratitude for their love, patience, and support throughout this process, especially my parents.
My former colleagues at Adaptive Path fed me a constant diet of new material, both in links to interesting stuff and in projects to work on that constantly expanded what I knew—or needed to know—to make this book as rich as it is.
This book would be boring indeed without the images contained therein, and I'd like to thank the companies that contributed them and the individuals who graciously made their photographs available via Creative Commons licenses for commercial use.
My colleague and friend, Rachel Glaves, created many of the illustrations in the book, and the book is much improved because of it.
My other colleague and friend, Sarah B. Nelson, took the photographs in the appendix and made them look good. My model, Ellen Ho, really brought the gestures to life.
I also benefited greatly from the review of others who read and commented on the manuscript as it was in development. For their advice and expertise, I thank them: Bill DeRouchey, Kevin Arthur, Nathan Moody, Juhan Sonin, Cem Keskin, Chong Lee Khoo, Dan Harrelson, Kate Fitch, and Jennifer Bove.
The contributors to the Interaction Design Association's mailing list and especially to the Interactive Gestures Wiki were an enormous help and source of material and inspiration.
I also want to thank the staff at O'Reilly who took a chance on this book and have supported its development and promotion: Jacque McIlvaine, Marlowe Shaeffer, Katheryn Barrett, Laurel Ackerman, Ron Bilodeau, Rachel Monaghan, Adam Witwer, Audrey Doyle, Jessamyn Read, Tim O'Reilly, and especially my e-ditor, Mary Treseler.
San Francisco, California
November 2008