"Sense, sure, you have, Else you could not have motion."
The following are patterns for gestures that are typically done in space, not via a touchscreen or other interactive surface. Of course, these patterns don't have to be mutually exclusive; free-form and touchscreen patterns can exist on the same device. For instance, a touchscreen kiosk may use Proximity Activates/Deactivates as a way to turn the kiosk on and off.
This pattern is for performing the simplest of all gestures, requiring only the presence of a person without any direct body contact (except perhaps to a pressure sensor). When a body is detected, an action triggers.
Use Proximity Activates/Deactivates to trigger simple on/off settings, such as lighting, display changes, sound, and other environmental controls.
Environmental and cost-saving concerns have driven the adoption of this pattern. For instance, if lights come on only when a person is in the room, it saves energy (and money).
The presence of a person can be detected with a variety of means: camera, motion detector, infrared "tripwire," pressure sensor, or microphone.
Of almost all the gestures, this is the one most likely to be triggered accidentally, so care should be taken when calibrating the sensor.
It may be beneficial in certain instances (for privacy or environmental controls) to be able to detect multiple people in one location, which requires considerably more nuance in processing the data.
A variation on this is to change the action based on other (somewhat) detectable characteristics, such as race, age, and gender.[27] For instance, a store advertisement that utilizes a camera could change its display based on the gender of a passerby.
Figure 4-1. Glowing Places combines plastic seating with LED strips and pressure sensors to measure the presence of people over time. Both the number of people sitting and the length of time they stay create lighting effects in the furniture. Many people sitting for brief periods of time result in more active lighting, whereas one or two people sitting for a longer period trigger mellow lighting. Courtesy Philips Lighting.
Figure 4-2. Swinxs is an outdoor game console for children. Several of the games, such as the racing and hide-and-seek games, involve getting close to the base station with the (included) wristband. For example, in the case of a racing game, Swinxs would use the wearer's proximity to the base station to determine the winner. Courtesy Swinxs BV.
The physical movement of the body or a part of the body (not just its presence as in Proximity Activates/Deactivates and not a specific gesture) triggers an action.
Use this when the action you are enabling requires that the person be moving, not just be in an area (as in Proximity Activates/Deactivates). This pattern is often found in alarms and other environmental systems.
Instead of Proximity Activates/Deactivates, use Move Body to Activate either because the movement itself is meaningful (e.g., an activity such as dancing or running) and the action triggered is related to that motion, or to prevent false triggering.
This pattern requires the ability to sense movement—frequently, directional movement. This can be accomplished with cameras or infrared beams, or via accelerometers embedded in wearable devices.
Figure 4-4. The Nike+iPod Sports Kit consists of an accelerometer that is attached to or embedded in a shoe, which communicates to an iPod Nano. Users can track in real time their calories burned, distance, or time while running. Courtesy Apple.
Pointing with a finger or an object either turns on (activates) an object or selects it for further manipulation.
Use Point to Select/Activate to select objects on-screen or in the environment so that you can interact with them by sending commands to them.
Point to Select/Activate requires the ability to detect the orientation of the user's hand. This can be accomplished via cameras or other means involving objects held in the hand (e.g., a remote control) or worn on the hand (e.g., a glove) that can indicate the position of the hand in relation to other objects (either in the environment or on-screen). In an environment, each object could have an individual sensor attached to it that detects when a hand is pointing at it (e.g., via a beam of infrared light).
Use Wave to Activate for a simple gesture that is not typically done by accident. As such, this pattern is good for use in public spaces.
Wave to Activate is a simple, common gesture. It is also used by people trying to manually trigger Move Body to Activate or Proximity to Activate.
A number of sensors can be employed to detect a wave, including a camera and, most frequently, an infrared beam. A wave can also be captured by a very small range on a motion detector or heat sensor.
Figure 4-10. To use enMotion's Paper Towel Dispenser, you wave your hand in front of the sensor, and a sheet of paper towel comes out of the dispenser. Courtesy Georgia-Pacific.
Use Place Hands Inside to Activate when you want something to affect the hands or you want to turn on controls that would be manipulated by the hands.
Hands are used in so many different ways, it was inevitable that some patterns would focus on them. This pattern is still emerging, however, and is mostly seen today in bathrooms for washing and drying hands.
Figure 4-13. The Tripoint Gooseneck Touchless Electronic Faucet releases water when hands are placed inside the sink. Courtesy Kohler.
Turning or tilting an object changes either the functionality of the object or the feedback/display of items on that object.
Use Rotate to Change State on devices that can be used for multiple purposes or to display content that might be better viewed in portrait or landscape view.
Particularly useful on devices with limited, rectangular screen real estate, Rotate to Change State allows content to be viewed in ways that best display the content, given the screen size.
Rotate to Change State requires a gyroscope or a multiaxis accelerometer to determine the position of the device. The threshold for the change in state (e.g., 90 degrees, 180 degrees, etc.) needs to be determined.
If the device lays flat (e.g., on a table), accelerometers may not pick up the rotation.
Figure 4-16. Nokia's newer N series phones, such as the N95 (pictured), are equipped with accelerometers and can change views from portrait to landscape based on how they are held. Courtesy Nokia.
Use Step to Activate when other proximity detectors (sound, infrared cameras, etc.) would be impractical, or for actions that would naturally be delivered by the feet (stepping, dancing, etc.).
This pattern is useful for alternative controls when the hands are occupied or for controls in low places where it would be difficult to reach with the hands.
Step to Activate typically requires a pressure sensor placed inside or beneath a flat object, sometimes to protect the sensor from hard stomps or presses.
Figure 4-19. One of the many incarnations of Dance Dance Revolution, where players follow on-screen cues to step in various directions quickly to play the game (and thus to dance). Courtesy Warren Rohner.
Use Shake to Change for very simple actions that have little nuance, usually with objects that can be held in the hand easily.
This is a simple action that most people can perform. Also, it has an element of fun and delight to it.
Shake to Change requires an accelerometer to detect motion. When enough vibration is detected, the action triggers.
Moving the body, limb, or hand in a particular direction moves an avatar, an on-screen object, or a physical object in that direction.
Use Tilt to Move as a means of controlling direction in a 3D or physical space. Note that body tilting may be difficult for young children or the elderly.
Tilting allows for a lot of nuanced control of movement, yet it also allows the body to remain mostly stationary while doing so.
Tilt to Move generally requires some sort of gyroscope or multiaxis accelerometer inside a device to measure angle, or it can be faked with pressure pads on an object that is being stood on, detecting changes in pressure/weight as the user tilts a limb or the body in a specific direction.
Figure 4-25. Designed for navigating 3D worlds, the SpaceNavigator has a cap that users can tilt (as well as twist and move from side to side and up and down) to move their avatar in any direction in a 3D environment or to manipulate 3D objects. Courtesy 3Dconnexion.
Figure 4-26. To operate a Segway, drivers tilt in the direction they want to go. Courtesy Segway, Inc.
Figure 4-27. The Wii Balance Board uses a series of pressure sensors to detect a user's balance to play games such as We Ski. Courtesy Nintendo.
The next chapter looks at documenting gestural systems—systems that may contain the patterns in this chapter and the preceding chapter.
[27] See, for instance, "Billboards That Look Back," by Stephanie Clifford, in the New York Times, May 31, 2008 (http://www.nytimes.com/2008/05/31/business/media/31billboard.html?ex=1370145600&en=c4fb20d8fbcba416&ei=5124&partner=permalink&exprod=permalink).