In 3D UIs, the situation is similar: there are some 3D interfaces where simple physical motions, such as walking, can be used for travel (e.g., when head and/or body trackers are used), but this is only effective within a limited space at a very limited speed. For most travel in 3D UIs, our actions must be mapped to travel in other ways, such as through a vehicle metaphor, for example. A major difference between real-world travel in vehicles and virtual travel, however, is that 3D UIs normally provide only visual motion cues, neglecting vestibular cues—this visual-vestibular mismatch can lead to cybersickness (see Chapter 3, “Human Factors Fundamentals,” section 3.3.2.)

Interaction techniques for the task of travel are especially important for two major reasons. First, travel is easily the most common and universal interaction task in 3D interfaces. Although there are some 3D applications in which the user’s viewpoint is always stationary or where movement is automated, those are the exception rather than the rule. Second, travel (and navigation in general) often supports another task rather than being an end unto itself. Consider most 3D games: travel is used to reach locations where the user can pick up treasure, fight with enemies, or obtain critical information. Counterintuitively, the secondary nature of the travel task in these instances actually increases the need for usability of travel techniques. That is, if the user has to think about how to turn left or move forward, then he has been distracted from his primary task. Therefore, travel techniques must be intuitive—capable of becoming “second nature” to users.

In virtual worlds, wayfinding can also be crucial. In a large, complex environment, an efficient travel technique is of no use if one has no idea where to go. When we speak of “wayfinding techniques,” we refer to designed wayfinding aids included as part of the interface or in the environment. Unlike travel techniques or manipulation techniques, where the computer ultimately performs the action, wayfinding techniques only support the performance of the task in the user’s mind (see Chapter 3, section 3.4.2 for a discussion of situation awareness, cognitive mapping, and the different types of spatial knowledge).

Clearly, travel and wayfinding are both part of the same process (navigation) and contribute towards achieving the same goals. However, from the standpoint of 3D UI design, we can generally consider them to be distinct. A travel technique is necessary to perform navigation tasks, and in some small or simple environments a good travel technique may be all that is necessary. In more complex environments, wayfinding aids may also be needed. In some cases, the designer can combine techniques for travel and wayfinding into a single integrated technique, reducing the cognitive load on the user and reinforcing the user’s spatial knowledge each time the technique is used. Techniques that make use of miniature environments or maps fit this description (see section 8.6.1), but these techniques are not suitable for all navigation tasks.

walking (section 8.4)

steering (section 8.5)

selection-based travel (section 8.6)

manipulation-based travel (section 8.7)

Section 8.8 completes the discussion of travel by discussing other aspects of travel technique design. The design of wayfinding aids is presented in section 8.9. Finally, we present design guidelines for navigation interfaces (section 8.10), followed by our case studies (section 8.11).

To what extent should 3D UIs support exploration tasks? The answer depends on the goals of the application. In some cases, exploration is an integral component of the interaction. For example, in a 3D visualization of network traffic data, the structure and content of the environment is not known in advance, making it difficult to provide detailed wayfinding aids. The benefits of the visualization depend on how well the interface supports exploration of the data. Also, in many 3D gaming environments, exploration of unknown spaces is an important part of the entertainment value of the game. On the other hand, in a 3D interface where the focus is on performing tasks within a well-known 3D environment, the interface designer should provide more support for search tasks via goal-directed travel techniques.

Naïve search has similarities with exploration, but clues or wayfinding aids may direct the search so that it is much more limited and focused than exploration. Primed search tasks also exist on a continuum, depending on the amount of knowledge the user has of the target and the surrounding environment. A user may have visited a location before but still might have to explore the environment around his starting location before he understands how to begin traveling toward the goal. On the other hand, a user with complete survey knowledge of the environment can start at any location and immediately begin navigating directly to the target. Although the lines between these tasks are often blurry, it is still useful to make the distinction.

Many 3D UIs involve search via travel. For example, the user in an architectural walkthrough application may wish to travel to the front door to check sight lines. Techniques for this task may be more goal oriented than techniques for exploration. For example, the user may specify the final location directly on a map rather than through incremental movements. Such techniques do not apply to all situations, however. Bowman, Johnson, and Hodges (1999) found that a map-based technique was quite inefficient, even for primed search tasks, when the goal locations were not explicitly represented on the map. It may be useful to combine a target-based technique with a more general technique to allow for the continuum of tasks discussed above.

A designer might consider maneuvering tasks to be search tasks, because the destination is known, and therefore use the same type of travel techniques for maneuvering as for search, but this would ignore the unique requirements of maneuvering tasks. In fact, some applications may require special travel techniques solely for maneuvering. In general, travel techniques for this task should allow great precision of motion but not at the expense of speed. The best solution for maneuvering tasks may be physical motion of the user’s head and body because this is efficient, precise, and natural, but not all applications include head and body tracking, and even those that do often have limited range and precision. Therefore, if close and precise work is important in an application, other techniques for maneuvering, such as the object-focused travel techniques in section 8.7.2, must be considered.

Distance to be traveled: In a 3D UI using head or body tracking, it may be possible to accomplish short-range travel tasks using natural physical motion only. Medium-range travel requires a virtual travel technique but may not require velocity control. Long-range travel tasks should use techniques with velocity control or the ability to jump quickly between widely scattered locations.

Amount of curvature or number of turns in the path: Travel techniques should take into account the amount of turning required in the travel task. For example, steering (see section 8.5.1) based on torso direction may be appropriate when turning is infrequent, but a less strenuous method, such as hand-directed steering (most users will use hand-directed steering from the hip by locking their elbows in contrast to holding up their hands), would be more comfortable when the path involves many turns.

Visibility of the target from the starting location: Many target-based techniques (section 8.6) depend on the availability of a target for selection. Gaze-directed steering (section 8.5.1) works well when the target is visible but not when the user needs to search for the target visually while traveling.

Number of DOF required for the movement: If the travel task requires motion only in a horizontal plane, the travel technique should not force the user to also control vertical motion. In general, terrain-following is a useful constraint in many applications.

Required accuracy of the movement: Some travel tasks require strict adherence to a path or accurate arrival at a target location. In such cases, it’s important to choose a travel technique that allows for fine control and adjustment of direction, speed, or target location. For example, map-based target selection (section 8.6) is usually inaccurate because of the scale of the map, imprecision of hand tracking, or other factors. Travel techniques should also allow for easy error recovery (e.g., backing up if the target was overshot) if accuracy is important.

Other primary tasks that take place during travel: Often, travel is a secondary task performed during another more important task. For example, a user may be traveling through a building model in order to count the number of windows in each room. It is especially important in such situations that the travel technique be unobtrusive, intuitive, and easily controlled.

active versus passive

physical versus virtual

using task decomposition

by metaphor

Direction or target selection refers to the primary subtask in which the user specifies how to move or where to move.

Velocity/acceleration selection describes how users control their speed.

Conditions of input refers to how travel is initiated, continued, and terminated.

This taxonomy covers a large portion of the design space for travel techniques (only a few representative technique components are shown in the figure) and allows us to view the task of travel in more fine-grained chunks (subtasks) that are separable. By choosing a technique component for each of the three subtasks, we can define a complete travel technique. For example, the most common implementation of the pointing technique (see section 8.5.1) uses pointing steering, constant velocity/acceleration, and continuous input (a button held down).

A second task decomposition (Bowman, Davis et al. 1999) subdivides the task of travel in a different, more chronological way (Figure 8.2). In order to complete a travel task, the user first starts to move, then indicates position and orientation, and then stops moving. Of course, this order does not always strictly hold: in some target-based techniques, the user first indicates the target position and then starts to move. One feature of this taxonomy that distinguishes it from the first is the explicit mention of specifying viewpoint orientation, which is an important consideration for 3D UIs without head tracking (see section 8.8.1). As shown in the figure, the taxonomy further decomposes the position specification subtask into position (xyz coordinates), velocity, and acceleration subtasks. Finally, it lists three possible metaphors for the subtask of specifying position: discrete target specification, one-time route specification, and continuous specification of position. These metaphors differ in the amount of control they give the user over the exact path (the active/passive distinction). We discuss these metaphors in sections 8.5 and 8.6.

In this chapter, we provide our own classification based on metaphors. Sections 8.4–8.7 organize travel techniques by four common metaphors: walking, steering, target/route selection, and travel-by-manipulation. These sections consider only the task of controlling the position of the viewpoint and do not consider issues such as specifying viewpoint orientation, controlling the speed of travel, or scaling the world. We cover these issues in section 8.8.

real walking

redirected walking

scaled walking

Real walking also raises issues with cabling: cables for trackers, input devices, and/or displays may not be long enough to allow complete freedom of movement in the tracked area, and unless cabling is carefully handled by another person or managed by a mechanical system, walking users can easily become tangled in cables as they move about the space. Wireless devices, such as wireless video transmitters and receivers, can alleviate these concerns for some systems. Alternatively, in so-called backpack systems, the user carries a portable computer to which all the devices are connected.

Usoh et al. (1999) used a large-area tracking system to determine the effect of a physical walking technique on the sense of presence. They created a small, compelling environment with a virtual pit whose bottom was far below the floor on which the user stood. They found that users who physically walked through the environment felt more present and exhibited greater fear of the virtual pit than users who walked in place (described in section 8.4.2) or used a virtual travel technique. Other researchers have found that real walking leads to a higher level of spatial knowledge in a complex environment (Chance et al. 1998).

Real walking is also becoming more important for other types of 3D applications, especially mobile augmented reality (Höllerer et al. 1999; Nurminen et al. 2011). In these systems, users are free to walk around in very large-area indoor or outdoor environments and have additional graphical information superimposed on their view of the real world (Figure 8.4). These applications often use Global Positioning System (GPS) data for tracking, possibly enhanced with orientation information from self-contained inertial trackers. Most GPS devices only give coarse-grained position information on the scale of meters, which is not sufficiently precise for most 3D UIs. Recent technological developments may lead to increasing the precision of most GPS devices to centimeters (Pesyna et al. 2014), however, most 3D UIs will still require millimeter precision. Another option is to use inside-out optical tracking of the environment using SLAM or one of its variants (see Chapter 6, “3D User Interface Input Hardware,” section 6.3.1).

Real walking can be a compelling travel technique, but its effectiveness is dependent upon the tracking system and tracking area. In most practical 3D UIs, real walking can be used in a small local area, but other techniques are needed to reach other parts of the environment. However, in our experience, when users are allowed to both walk and use a virtual travel technique, they quickly learn to use only the virtual technique because it requires less effort.

A basic approach to redirected walking is the stop-and-go method (Bruder et al. 2015). With this method, the virtual world is rotated around a stationary user faster or slower than the user is physically turning (i.e., rotation gains are applied). This allows the technique to orient the user’s forward direction away from the tracking boundaries. The key to the stop-and-go method is to influence the user to physically turn in the first place. Researchers have investigated a number of ways to accomplish this, including location-oriented tasks (Kohli et al. 2005), visual distractors (e.g., a butterfly; Peck et al. 2009), and verbal instructions (Hodgson et al. 2014).

A more advanced approach to redirected walking is to continuously redirect the user while walking through the virtual environment (Bruder et al. 2015). For example, if the user is walking straight ahead in the virtual world, small rotations of the virtual scene can be used to redirect the user along a circular path within the tracking area (Figure 8.5). If the rotations are small enough, they will be imperceptible and the user will have the impression of truly walking straight ahead in the virtual world.

When implementing a redirected walking technique, it is important to avoid perceptible visual conflicts with the proprioceptive and vestibular cues. Steinicke et al. (2010) conducted studies to estimate detection thresholds for these conflicts. They found that users could be physically turned about 49% more or 20% less than a displayed virtual rotation. Users can also be physically translated 14% more or 26% less than a displayed virtual translation. Additionally, the researchers found that users will perceive themselves walking straight in the virtual environment while being continuously redirected on a circular arc if the arc’s radius is greater than 22 meters.

An important aspect of continuous redirected walking is how to rotate the virtual scene relative to the physical tracking space while the user is walking. Two commonly used algorithms are steer-to-center and steer-to-orbit (Hodgson et al. 2014). The goal of the steer-to-center algorithm is to continuously redirect the user’s physical movements through the center of the tracking area while adhering to the detection thresholds. In contrast, the goal of the steer-to-orbit algorithm is to continuously redirect the user’s physical movements along a circular arc, as if the user were orbiting the center of the tracking space. Hodgson and Bachmann have shown that the steer-to-center algorithm helps to avoid tracking boundaries better for relatively open-spaced virtual environments (Hodgson and Bachmann 2013), while the steer-to-orbit algorithm performs better for relatively enclosed VEs (Hodgson et al. 2014).

An alternative to rotating the virtual scene is to manipulate the dimensional properties of the scene. Suma et al. (2012) demonstrated that VEs can make use of self-overlapping architectural layouts called “impossible spaces.” These layouts effectively compress large interior environments into smaller tracking spaces by showing only the current room to the user despite another room architecturally overlapping it. Vasylevska et al. (2013) have expanded upon the concept of impossible spaces to create flexible spaces, which are created by dynamically generating corridors between rooms to direct the user away from the center of the tracking space when exiting a room and back toward the center when entering the next room.

While redirected walking can overcome some of the tracking limitations of real walking, it has its own challenges. First, continuous redirected walking requires a large tracking space to be imperceptible. Using it in smaller spaces is not very effective. Second, even in large tracking spaces, users can still walk outside of the space if they decide to ignore visual cues, tasks, and distractors. Finally, Bruder et al. (2015) have demonstrated that redirected walking with small arcs demands more cognitive resources than with larger arcs. For those readers interested in more details on redirected walking, we recommend a book on human walking in VEs (Steinicke et al. 2013).

A better scaled-walking approach is the Seven League Boots technique (Interrante et al. 2007). With this technique, only the user’s intended direction of travel is scaled. This avoids scaling the oscillatory motions of the user’s head that occur during gait, which would result in excessive swaying of the user’s viewpoint. Seven League Boots also uses a dynamically changing scaling factor based on the speed of physical walking. When users walk slowly, no scaling is applied, but at faster walking speeds, travel is scaled to be even faster. This is similar to common mouse acceleration techniques in desktop UIs and enables users to explore local areas with one-to-one walking and reach distant areas by quickly walking in their direction.

The user’s intended direction of travel must be determined to implement Seven League Boots. Interrante et al. (2007) propose that the best method is to determine the direction of travel as a weighted combination of the user’s gaze direction and the direction of the user’s horizontal displacement during the previous few seconds. When displacement is minimal, as it is when standing, a weight of 1 is assigned to the gaze direction. However, when displacement resembles walking, a weight of 0 is assigned to the gaze direction to allow the user to look around while walking.

While scaled walking techniques afford the biomechanics of a full gait cycle and circumvent the physical limitations of real walking, they can be unnatural to use. Based on work by Steinicke et al. (2010), scaled walking techniques are perceptible when the scale factor is greater than 1.35 times the user’s physical motions. And while scaled walking can increase the size of the area users can reach with natural walking, that area is still finite, meaning that additional travel techniques will be needed to reach other parts of the virtual environment.

walking in place

human joystick

However, there are several caveats to be considered. First, walking in place does not incorporate the swing phase of the gait cycle, as real walking does, so the sense of presence or real movement is diminished. Second, the size of the environment, while theoretically unlimited, still has a practical limitation because users exert more energy while using walking in place than real walking (Nilsson et al. 2013).

Several approaches have been used to implement walking in place. Early on, Slater et al. (1995) used a position tracker and a neural network to analyze the motions of the user’s head to distinguish walking in place from other types of movements. When the system detects that the user is walking in place, it moves the user through the virtual world in the direction of the user’s gaze. Instead of tracking head motions, Templeman et al. (1999) tracked and analyzed leg motions in their Gaiter system to determine when the user was stepping in place and in which direction. In a similar system, Feasel et al. (2008) used a chest tracker and the orientation of the user’s torso to determine the direction of virtual motion. They also analyzed vertical heel movements instead of steps to reduce the half-step latency seen in most other implementations.

More recently, Nilsson et al. (2013) investigated the kinematics of walking in place (Figure 8.6). In most implementations, a marching gesture is utilized, in which the user lifts each foot off the ground by raising the thighs. Nilsson et al. (2013) proposed two additional gestures—wiping and tapping. With wiping, the user bends each knee while keeping the thigh relatively steady, similar to wiping one’s feet on a doormat. With tapping, the user lifts each heel off the ground while keeping the toes in contact with the ground. In a study of perceived naturalness, the researchers found that tapping was considered the most natural and least strenuous of the three walking-in-place gestures.

Studies have shown that walking in place does increase the sense of presence relative to completely virtual travel but that it’s not as effective as real walking (Usoh et al. 1999). These techniques also sometimes suffer from the problems of recognition errors and user fatigue. This approach applies to systems where higher levels of naturalism are needed and where the environment is not too large. For applications in which the focus is on efficiency and task performance, however, a steering technique (see section 8.5) is often more appropriate.

Another implementation of the human joystick technique is to calculate the 2D horizontal vector from the center of the tracking space to the user’s tracked head position and then use that vector to define the direction and velocity of virtual travel. McMahan et al. (2012) implemented this technique in a CAVE-like system. They defined a no-travel zone at the center of the tracking volume to avoid constant virtual locomotion due to minute distances between the user’s head position and the center of the tracking volume. To initiate travel, the user stepped outside of the neutral zone to define the 2D travel vector. After this initial swing phase, the user simply stood in place to continue the same direction and speed of travel. The user could move closer to or farther from the center to adjust the travel speed. Additionally, the user could move laterally to redefine the direction of travel relative to the tracking center.

In both implementations of the human joystick, the user’s gaze direction is independent of the direction of travel, which is defined by either the positions of the user’s feet or the position of the user’s head. This allows the user to travel backwards, which techniques like walking in place do not allow. The human joystick has also been shown to afford higher levels of presence (McMahan et al. 2012) than purely virtual travel. However, McMahan (2011) also found that the human joystick technique was significantly less efficient than a traditional keyboard and mouse travel technique. He attributed this to the fact that the human joystick technique requires the user to step back to the center of the tracking volume to terminate travel. This is very different from real human gait, which requires little to no effort to stop locomotion.

treadmills

passive omnidirectional treadmills

active omnidirectional treadmills

low-friction surfaces

step-based devices

As explained, passive omnidirectional treadmills rely on the user’s weight and momentum to activate their surfaces. In turn, these moving surfaces negate the user’s gait and keep the user in the center of the space. An example of a passive omnidirectional treadmill is the Omnidirectional Ball-Bearing Disc Platform (OBDP) developed by Huang (2003). This prototype uses hundreds of ball bearings arranged in a concave platform to passively react to the user’s walking motions. When the user steps forward, the ball bearings roll toward the center of the device due to the curvature and return the user’s foot to the center of the platform. Due to the unnaturalness of stepping on a concave surface and the abrupt response of the ball bearings, the OBDP also requires a frame around the user’s waist to stabilize the user in the event of a fall.

Another example of a passive omnidirectional treadmill is the Virtusphere (Figure 8.8; Medina et al. 2008). This device serves as a human-sized “hamster ball” that rolls in place while the user walks inside of it. Like the OBDP, the weight of the user’s steps initiates the rolling of the Virtusphere’s surface. However, once the Virtusphere is rolling, the user must exert reverse forces in order to stop the device without falling. Research conducted by Nabiyouni et al. (2015) shows that the Virtusphere provides a user experience that is significantly worse than a joystick or real walking.

The most important issue was that because the ODT continually moved the surface in order to recenter the user on the treadmill and keep the user from walking off the edge of the device, the recentering motion caused the user to lose his or her balance, especially during turns or sidestepping. The ODT was found to support walking at a moderate speed with gradual turns, but not many of the other movements that soldiers would need to make.

Another approach to creating an active omnidirectional treadmill is to use conveyor rollers in a radial pattern (i.e., with all the rollers’ direction of motion being toward the center of the device) instead of belts. Some commercial versions consist of 16 sections of rollers surrounding a small stable platform in the center. When the user is detected walking outside of the small platform by an optical tracking system, the relevant rollers are actively controlled to move the user back to the center platform. The speed of the rollers is controlled to match the user’s forward momentum. Otherwise, the user may fall if there’s a mismatch in inertia and roller speed.

There are several fundamental challenges for low-friction surfaces. First, the degree of friction must be properly balanced between the surface and the shoes. If the amount of friction is too high, the user will exert much more energy to move than in real walking. If the amount of friction is too low, the surface will be slippery, and the user is likely to slip or fall. Second, though these devices are designed to simulate real walking, they require biomechanics that are more similar to skating than walking. Finally, based on the authors’ personal experiences, low-friction surfaces require time to learn how to use effectively.

Another step-based device example is the CirculaFloor, also developed by Iwata et al. (2005). This system uses multiple omnidirectional robotic vehicles as moveable tiles that are programmed to position themselves under the user’s steps. Each tile provides a sufficient area for walking to avoid requiring precise positioning. As the user walks, the vehicles can move in the opposite direction to cancel the user’s global movement. Additionally, once the user steps off a tile, it can be circulated back to catch the next step.

In general, gait negation devices have not been as successful as one might hope. This is because they are still too expensive, too susceptible to mechanical failures, and too slow to respond to the user’s movements. Most importantly, the devices do not produce the perception of natural walking for the user because of biomechanics and forces that are different from real walking. Therefore, instead of being able to use her natural ability to walk, the user must learn to adapt her walking motion to the device’s characteristics. Still, such devices may prove useful in certain applications that require physical locomotion via walking. For a more detailed overview of locomotion devices, see Hollerbach (2002) and Steinicke et al. (2013).

gaze-directed steering

hand-directed steering (pointing)

torso-directed steering

lean-directed steering

The basic concept of gaze-directed steering can be extended by allowing motion along vectors orthogonal to the gaze vector. This gives the user the ability to strafe—to move backward, up, down, left, and right (Chee and Hooi 2002). This ability is especially important for desktop systems, where setting the gaze direction may be more cumbersome than in a head-tracked VE. Additionally, strafing is often used for maneuvering (see section 8.2.3).

From the user’s point of view, gaze-directed steering is quite easy to understand and control. In a desktop 3D environment, it seems quite natural to move “into” the screen. In an immersive 3D UI with head tracking, this technique also seems intuitive, especially if motion is constrained to the 2D horizontal plane. In addition, the hardware requirements of the technique are quite modest; even in an immersive system, the user needs only a head tracker and a button. However, if complete 3D motion is provided (i.e., “flying”), gaze-directed steering has two problems. First, when users attempt to travel in the horizontal plane, they are likely to travel slightly up or down because it’s very difficult to tell whether the head is precisely level. Second, it’s quite awkward to travel vertically up or down by looking straight up or down, especially when wearing an HWD.

But perhaps the most important problem with gaze-directed steering is that it couples gaze direction and travel direction, meaning that users cannot look in one direction while traveling in another. This may seem a small issue, but consider how often you look in a direction other than your travel direction while walking, cycling, or driving in the physical world. Studies have shown that pointing (see below) outperforms gaze-directed steering on tasks requiring motion relative to an object in the environment (Bowman et al. 1997).

An extension of the pointing concept uses two hands to specify the vector (Mine 1997). Rather than use the orientation of the hand to define the travel direction, the vector between the two hands’ positions is used. An issue for this technique is which hand should be considered the “forward” hand. In one implementation using Pinch Gloves (Bowman, Wingrave et al. 2001), the hand initiating the travel gesture was considered to be forward. This technique makes it easy to specify any 3D direction vector and also allows easy addition of a velocity-control mechanism based on the distance between the hands.

This pointing technique is more flexible but also more complex than gaze-directed steering, requiring the user to control two orientations simultaneously. This can lead to higher levels of cognitive load, which may reduce performance on cognitively complex tasks like information gathering (Bowman, Koller et al. 1998). The pointing technique is excellent for promoting the acquisition of spatial knowledge because it gives the user the freedom to look in any direction while moving (Bowman, Davis et al. 1999).

The major advantage of the torso-directed technique is that, like pointing, it decouples the user’s gaze direction and travel direction. Unlike pointing, it does this in a natural way. The user’s cognitive load should be lessened with the torso-directed technique, although this has not been verified experimentally. The torso-directed technique also leaves the user’s hands free to perform other tasks. However, the torso-directed technique also has several disadvantages. The most important of these is that the technique applies only to environments in which all travel is limited to the horizontal plane, because it is not practical to point the torso up or down. The technique also requires an additional tracker beyond the standard head and hand trackers for tracking the user’s torso.

One example is the PenguFly technique developed by von Kapri et al. (2011). With PenguFly, both of the user’s hands are tracked in addition to the head. The direction of travel is then specified by the addition of the two vectors defined from each hand to the head. The travel speed is defined by half of the length of this lean-directed vector (Figure 8.11). Lean-directed steering has been shown to be more accurate for traveling than pointing due to its higher granularity of control over the travel speed (von Kapri et al. 2011). However, it also induces a significant increase in cybersickness, likely due to the large body motions required.

Another lean-directed steering implementation is the ChairIO interface developed by Beckhaus et al. (2007). The interface is based on an ergonomic stool with a rotating seat and a spring-based column that can tilt and even allow the user to bounce up and down. With magnetic trackers attached to the seat and the column, the ChairIO allows the movements of the user to be tracked as she leans and turns in the seat. This tracking information can then be used for steering. Similarly, Marchal et al. (2011) developed the Joyman interface, which consisted of essentially a trampoline, a rigid surface, an inertial sensor, and a safety rail to prevent falling. While holding onto the rail, the user could lean extremely far in any direction, which would cause the rigid surface to also lean toward one side of the trampoline’s structure. The inertial sensor detects the orientation of the lean and then translates it into a direction and speed for steering.

All of the lean-directed steering techniques integrate direction and speed into a single, easy-to-understand movement. These techniques allow the user to rely on natural proprioceptive and kinesthetic senses to maintain spatial orientation and understanding of movement within the environment. Additionally, Kruijff et al. (2016) found that adding walking-related sensory cues, such as footstep sounds, head-bobbing camera motions, and footstep-based vibrotactile feedback, can all enhance the user’s senses of vection and presence. The major disadvantage to these techniques is that they are limited to 2D locomotion unless another technique or feature is added to allow for vertical motion.

We discuss the following types of steering props in this section

cockpits

cycles

Other specialized steering props can be used to simulate real or imaginary vehicles for particular application domains. For example, realistic ship controls can be used to pilot a virtual ship (Brooks 1999), or an actual cockpit from a tractor can be used to control a virtual tractor (Deisinger et al. 2000). Of course, this near-field haptics approach (Brooks 1999) has been used in aircraft simulators for years. Disney used steering props in several of its VR-based attractions (Pausch et al. 1996; Schell and Shochet 2001). For example, the virtual jungle cruise ride at DisneyQuest in Orlando allows several guests to collaboratively steer and control the speed of a virtual raft using physical oars, and the Pirates of the Caribbean attraction includes a steering wheel and throttle for the virtual ship. In addition, many arcade games use such props—motorcycle handlebars, steering wheels, skateboards, skis, and the like.

The Uniport is a unicycle-like device that allows travel through virtual worlds seen through an HWD (Figure 8.12); it was designed for infantry training (as one of the precursors to the omnidirectional treadmill). It is obviously less effective at producing a believable simulation of walking, and users may also have difficulty steering the device. However, it is mechanically much less complex and produces significant exertion for the user, which may be desired for some applications.

Even though the user is concerned only with the target of travel, this should not necessarily be construed to mean that the system should move the user directly to the target via teleportation. An empirical study (Bowman et al. 1997) found that teleporting instantly from one location to another in a 3D UI significantly decreases the user’s spatial orientation (users find it difficult to get their bearings when instantly transported to the target location). Therefore, continuous movement from the starting point to the endpoint is recommended when spatial orientation is important. On the other hand, continuous movement that’s not under the user’s control can increase cybersickness. For this reason, a compromise is often used in which the user is not teleported instantaneously but instead is moved very quickly through virtual space to the target. This “blink” mode of travel gives users enough visual information to help them understand how they have moved, but is so short that it’s unlikely to make users feel sick.

There are many ways to specify the target of travel. In this section, we describe two techniques:

representation-based target techniques

dual-target techniques

Many other target-based travel techniques specify the target using interaction techniques designed for another task. We call this type of technique a cross-task technique because the technique implementation has crossed from one task to another. Cross-task target-specification techniques include

selecting an object in the environment as the target of travel using a selection technique (see Chapter 7, “Manipulation”)

placing a target object in the environment using a manipulation technique (see Chapter 7)

selecting a predefined target location from a list or menu (see Chapter 9, “System Control”)

entering 2D or 3D coordinates using a number entry technique or a location name using a text entry technique (see Chapter 9)

Zeleznik and colleagues used this technique in the context of a virtual museum. The technique allowed users to select a painting on the wall, examine that painting up close, and then return to the original location where multiple paintings could be viewed. Their implementation used a specialized pop-through button device (see Chapter 6, “3D User Interface Input Hardware”). Users moved to the target object with light pressure on the button and could choose either to remain there by pressing the button firmly or to return to their previous location by releasing the button. This technique could also be implemented using two standard buttons, but the pop-through buttons provide a convenient and intuitive way to specify a temporary action that can then be either confirmed or canceled. This general strategy could be applied to other route-planning and target-based techniques as well.

Route-planning techniques also allow the user to focus on other tasks, such as information gathering, during the actual period of travel. Route-planning techniques still give users at least some degree of control over their motion, but they move that control to the beginning of the travel task. This section contains information on the following route-planning techniques:

drawing a path

marking points along a path

In an immersive 3D UI, the user likely cannot reach the entire environment to draw in it directly, so drawing on a 2D or 3D map of the environment could be used to specify the path. This requires a transformation from the map coordinate system to the world coordinate system, and in the case of a 2D map, an inferred height.

Manipulation-based travel techniques for travel should be used in situations where both travel and object manipulation tasks are frequent and interspersed. For example, consider an interior layout application, where the user’s goal is to place furniture, carpets, paintings, and other items in a virtual room. This task involves frequent object manipulation tasks to place or move virtual objects and frequent travel tasks to view the room from different viewpoints. Moreover, the designer will likely move an object until it looks right from the current viewpoint and then travel to a new viewpoint to verify the object’s placement. If the same metaphor can be used for both travel and object manipulation, then the interaction with this environment will be seamless and simple from the user’s point of view.

camera manipulation

avatar manipulation

fixed-object manipulation

The camera-in-hand technique is relatively easy to implement. It simply requires a transformation between the tracker coordinate system and the world coordinate system, defining the mapping between the virtual camera position and the tracker position.

This technique can be effective for desktop 3D UIs (if a tracker is available) because the input is actually 3D in nature, and the user can use her proprioceptive sense to get a feeling for the spatial relationships between objects in the 3D world. However, the technique can also be confusing because the user has an exocentric (third-person) view of the workspace, but the 3D scene is usually drawn from an egocentric (first-person) point of view.

Similarly, a path can be defined by moving a user icon on a 2D map of the environment, a technique that would apply equally well to desktop and immersive 3D UIs. Because this path is only 2D, the system must use rules to determine the height of the user at every point along the path. A common rule would keep the viewpoint at a fixed height above the ground.

Let us consider a specific example of fixed-object manipulation in 3D UIs. Pierce et al. (1997) designed a set of manipulation techniques called image-plane techniques (see Chapter 7, section 7.5.1). Normally, an object is selected in the image plane, then hand movements cause that object to move within the environment. For example, moving the hand back toward the body would cause the selected object to move toward the user as well. When used in travel mode, however, the same hand motion would cause the viewpoint to move toward the selected object. Hand rotations can also be used to move the viewpoint around the selected object. The scaled-world grab technique (Mine et al. 1997) and the LaserGrab technique (Zeleznik et al. 2002) work in a similar way.

Fixed-object manipulation techniques provide a seamless interaction experience in combined travel/manipulation task scenarios, such as the one described above. The user must simply be aware of which interaction mode is active (travel or manipulation). Usually, the two modes are assigned to different buttons on the input device. The two modes can also be intermingled using the same selected object. The user would select an object in manipulation mode, move that object, then hold down the button for travel mode, allowing viewpoint movement relative to that selected object, then release the button to return to manipulation mode, and so on.

single-point world manipulation

dual-point world manipulation

In order to integrate this travel technique with an object manipulation technique, the system must simply determine whether or not the user is grabbing a moveable object at the time the grab gesture is initiated. If an object is being grabbed, then standard object manipulation should be performed; otherwise, the grab gesture is interpreted as the start of a travel interaction. In this way, the same technique can be used for both tasks.

Although this technique is easy to implement, developers should not fall into the trap of simply attaching the world object to the virtual hand, because this will cause the world to follow the virtual hand’s rotations as well as translations, which can be quite disorienting. Rather, while the grab gesture is maintained (or the button held down), the system should measure the displacement of the virtual hand each frame and translate the world origin by that vector.

In its simplest form, this technique requires a lot of arm motion on the part of the user to travel significant distances in the VE. Enhancements to the basic technique can reduce this. First, the virtual hand can be allowed to cover much more distance using a technique such as Go-Go (Poupyrev et al. 1996). Second, the technique can be implemented using two hands instead of one, as discussed next.

Another advantage of dual-point manipulation is the ability to also manipulate the view rotation while traveling. When the user has the world grabbed with both hands, the position of the user’s nondominant hand can serve as a pivot point while the dominant hand defines a vector between them. Rotational changes in this vector can be applied to the world’s transformation to provide view rotations in addition to traveling using dual-point manipulations. Additionally, the distance between the two hands can be used to scale the world to be larger or smaller, as discussed in section 8.8.5.

head tracking

orbital viewing

nonisomorphic rotation

virtual sphere techniques

A number of different nonisomorphic mappings could be used for setting the virtual viewpoint orientation. For a CAVE-like display, LaViola et al. (2001) used a nonlinear mapping function, which kicks in only after the user has rotated beyond a certain threshold, dependent on the user’s waist orientation vector and position within the CAVE. A scaled 2D Gaussian function has been shown to work well. LaViola et al. (2001) offer more details on nonisomorphic viewpoint rotation.

The user can control velocity in many different ways. We discuss the following approaches to defining or manipulating velocity:

discrete changes

continuous control

direct input

automated velocity

Other methods for continuously controlling velocity make use of physical devices. A physical prop that includes acceleration and braking pedals can be used to control velocity similar to controlling the speed of a vehicle. Velocity may also be controlled with the use of physical force-based devices, such as a force-feedback joystick. In these cases, velocity is a function of the amount of force applied. For more information on such techniques, see MacKenzie (1995).

An obvious advantage to the continuous control techniques is that the user can achieve his desired travel speed. However, with some implementations, such as the techniques based on relative positions, it can be difficult to sustain a specific speed.

Slater, Usoh, and Steed (1994) used the walking-in-place technique and collisions with virtual ladders and stairs to provide vertical travel via climbing. The direction of climbing was determined by whether the user’s feet collided with the bottom step of a ladder or staircase (climbing up) or with the top step (climbing down). While on a ladder or staircase, the user could reverse the climbing direction by physically turning around.

More recently, researchers have developed other techniques for climbing ladders. Takala and Matveinen (2014) created a technique based on reaching for and grabbing the rungs of a virtual ladder. Once a rung is grabbed, users can travel up or down by moving the grabbed rung in the opposite direction (e.g., moving the rung down to climb up). To provide a more realistic technique for climbing ladders, Lai et al. (2015) developed the march-and-reach technique, in which the user marches in place to virtually step on lower ladder rungs while reaching to virtually grab higher rungs. While users found the technique more realistic, it was more difficult to use than walking in place or Takala’s technique of reaching and grabbing.

The basic concept of semiautomated travel is that the system provides general constraints and rules for the user’s movement, and the user is allowed to control travel within those constraints. This idea is of course applicable to both immersive and desktop 3D UIs. A particular implementation of this concept is Galyean’s river analogy (Galyean 1995). He used the metaphor of a boat traveling down a river. The boat continues to move with the current whether the user is actively steering or not, but the user can affect the movement by using the rudder. In particular, he designed an application in which the designer defined a path through the environment (the river). The user was “attached” to this path by a spring and could move off the path by some amount by looking in that direction (Figure 8.18).

There are several challenges when designing a technique for scaling the world and traveling. One is that the user needs to understand the scale of the world so that he can determine how far to move and can understand the visual feedback he gets when he moves. Use of a virtual body (hands, feet, legs, etc.) with a fixed scale is one way to help the user understand the relative scale of the environment. Another issue is that continual scaling and rescaling may hasten the onset of cybersickness or discomfort (Bowman, Johnson et al. 2001). In addition, scaling the world down so that a movement in the physical space corresponds to a much larger movement in the virtual space will make the user’s movements much less precise.

There are two common approaches to designing scaling-and-traveling techniques:

active scaling

automated scaling

MSVEs allow the user to concentrate on traveling instead of scaling while still gaining the benefits of having the world scaled up or down. However, such VEs require careful design, as the hierarchy of objects and scales need to be intuitive and usable for the user.

An early novel approach to transitioning among travel modes was Zeleznik and Forsberg’s (1999) UniCam, a 2D camera-control mechanism (Figure 8.20) originally derived from the SKETCH camera-control metaphor (Zeleznik et al. 1996). In UniCam, the user controls travel through the 3D scene by making 2D gestural commands, using a mouse or stylus with a button, to manipulate the virtual camera. To facilitate camera translation and orbiting, the viewing window is broken up into two regions, a rectangular center region and a rectangular border region. If the user clicks within the border region, a virtual sphere rotation technique is used to orbit the viewer about the center of the screen. If the user clicks within the center region and drags the pointer along a horizontal trajectory, image-plane translation is invoked. If the user drags the pointer along a vertical trajectory, the camera zooms in or out. The user can also invoke orbital viewing about a focus point by making a quick click to define a dot and then clicking again elsewhere.

A similar but more recent approach to integrating travel modes is the Navidget travel widget (Hachet et al. 2008). With the Navidget technique, the user can first zoom in by encircling, with a pointer or stylus, the area to keep within the camera’s field of view. If the user holds the input button after sketching a circle, instead of zooming, a virtual sphere appears to provide orbital viewing. Upon release of the input button, the camera moves to the final viewing position. Four additional widgets are attached to the virtual sphere and can manipulate the size of the sphere, which in turn controls the viewing distance that the camera is placed at. Finally, an outer region allows the user to orbit between the front and back of the virtual sphere by moving into the region and then back into the sphere.

Multi-camera techniques have also been used with AR. Veas and colleagues (2010) investigated techniques for observing video feeds from multiple cameras within an outdoor environment. They developed three techniques for transitioning to remote camera feeds with visible viewpoints of the same scene—the mosaic, the tunnel, and the transitional techniques. The mosaic and tunnel techniques provide egocentric transitions to the remote camera while the transitional technique provides an exocentric transition. Veas et al. (2012) later developed a multiview AR system by combining different views of remote cameras, views generated by other users’ devices, a view of 2D optical sensors, cameras on unmanned aerial vehicles, and predefined virtual views—more on these techniques can be found in the mobile AR case study below (section 8.11.2).

In a different multi-camera approach, Sukan et al. (2012) stored snapshots of augmented scenes that could later be virtually visited without physically returning to their relative locations. Along with still images of the real world, the researchers also stored the position of the camera. This allowed for virtual augmentations and objects to be dynamically updated when the snapshots were virtually viewed later. This approach enabled users to quickly switch between viewpoints of the current scene.

field of view

motion cues

multisensory output

presence

search strategies

Another negative side effect of a small FOV is the lack of optical-flow fields in users’ peripheral vision. Peripheral vision provides strong motion cues, delivering information about the user’s direction, velocity, and orientation during movement. In addition, it has been shown that small FOVs may lead to cybersickness (Stanney et al. 1998).

The effect of vestibular cues on the orientation abilities of users in VEs has been the subject of a range of studies. Usoh et al. (1999) compared real walking (section 8.4.1) against walking in place (section 8.4.2) and hand-directed steering (section 8.5.1). The two walking metaphors, which included physical motions, were found to be better than the steering technique, which only included virtual motions. Other studies of virtual and physical travel have shown positive effects of real motion on spatial orientation (Klatzky et al. 1998; Chance et al. 1998).

Our understanding of the proper balance between visual and vestibular input is still being formed. Harris et al. (1999) performed tests matching visual and vestibular input and concluded that developers should support vestibular inputs corresponding to at least one-quarter of the amount of visual motion. However, statically tilting the user’s body has been shown to have a positive effect on self-motion perception in some cases. In a small study, Nakamura and Shimojo (1998) found that vertical self-motion increases as the user is tilted further back in a chair. In a more recent study, Kruijff et al. (2015) found that statically leaning forward increased perceived distances traveled, likely due to an increase in perceived speed of self-motion. Additionally, Kruijff et al. (2016) found that dynamically leaning enhanced self-motion sensations.

Using a search strategy inspired by navigation experts can increase its effectiveness. For example, search patterns such as those used during aviation search-and-rescue missions may aid a user during wayfinding (Wiseman 1995). The basic line search follows a pattern of parallel lines. The pattern search starts at a specific central point and moves further away from it, using quadratic or radial patterns. The contour search is designed to follow contours in a landscape, like a river or a mountain. Finally, the fan search starts from a center point and fans out in all directions until the target is found. Of course, the use of these search strategies is dependent on the content of the environment—they might work well in a large outdoor environment but would not make sense in a virtual building.

Another important search strategy is to obtain a bird’s-eye view of the environment rather than performing all navigation on the ground. Users can be trained to employ this strategy quite easily, and it results in significantly better spatial orientation (Bowman et al. 1999). This can even be automated for the user. In the “pop-up” technique (Darken and Goerger 1999), users can press a button to temporarily move to a significant height above the ground, and then press the button again to go back to their original location on the ground.

We assume that novice users can learn search techniques, even if the described pattern search strategies are seen primarily in expert navigators. Placing a rectangular or radial grid directly in the environment provides a visual path along which users can search. Although these grids may supply directional and depth cues, they do not force users to employ a good search strategy.

environment legibility

landmarks

maps

compasses

signs

trails

reference objects

With all this added flexibility come some difficult design choices, as clutter and confusion should be avoided. In general, it is recommended to use you-are-here markers and to show an up-to-date version of the environment on the map. However, designers should be careful about automatically rotating or zooming the map, as this can cause confusion, depending on the user’s task, or even inhibit the formation of survey knowledge (Darken and Cevik 1999).

Another design choice is where and when to display the map. On small AR or VR displays, a legible map may take up a large percentage of the display area. At the least, a mechanism to display or hide the map should be provided. One effective technique is to attach the virtual map to the user’s hand or a handheld tool, so that the map can be viewed or put away with natural hand motions. Figure 8.21 illustrates a map attached to the user’s hand in this way. Unlike most maps, this map is not a spatial representation of the 3D environment, but rather a hierarchical representation showing objects at different levels of scale (Bacim et al. 2009).

The authors are of the opinion that no set of 3D interaction techniques is perfect for all applications. Designers must carefully consider the travel tasks that will be performed in the application (section 8.2), what performance is required for travel tasks, in what environment travel will take place, and who will be using the application.

Example: A visualization of a molecular structure uses an exocentric point of view and has a single object as its focus. Therefore, an object-inspection technique such as orbital viewing is appropriate.

Many designers start with the assumption that the 3D interface should be as natural as possible. This may be true for certain applications that require high levels of realism, such as military training, but many other applications have no such requirement. Nonisomorphic “magic” travel techniques may prove much more efficient and usable.

Example: The task of furniture layout for an interior design application requires multiple viewpoints but not natural travel. A magic technique such as scaled-world grab (section 8.8.5) will be efficient and will also integrate well with the object manipulation task.

The travel technique cannot be chosen separately from the hardware used in the system.

Example: If a personal surround-screen display is used, a vehicle-steering metaphor for travel fits the physical characteristics of this display. If a pointing technique is chosen, then an input device with clear shape and tactile orientation cues, such as a stylus, should be used instead of a symmetric device, such as a ball.

Similarly, the travel technique cannot be isolated from the rest of the 3D interface. The travel technique chosen for an application must integrate well with techniques for selection, manipulation, and system control.

Example: A travel technique involving manipulation of a user representation on a 2D map suggests that virtual objects could be manipulated in the same way, providing consistency in the interface.

Many applications include a range of travel tasks. It is tempting to design one complex technique that meets the requirements for all these tasks, but including multiple simple travel techniques is often less confusing to the user. Some VR applications already do this implicitly, because the user can physically walk in a small local area but must use a virtual travel technique to move longer distances. It will also be appropriate in some cases to include both steering and target-based techniques in the same application. We should note, however, that users will not automatically know which technique to use in a given situation, so some small amount of user training may be required.

Example: An immersive design application in which the user verifies and modifies the design of a large building requires both large-scale search and small-scale maneuvering. A target-based technique to move the user quickly to different floors of the building plus a low-speed steering technique for maneuvering might be appropriate.

If the user’s goal for travel is not complex, then the travel technique providing the solution to that goal should not be complex either. If most travel in the environment is from one object or well-known location to another, then a target-based travel technique is most appropriate. If the goal is exploration or search, a steering technique makes sense.

Example: In a virtual tour of a historical environment, the important areas are well known, and exploration is not required, so a target-based technique such as choosing a location from a menu would be appropriate. In a visualization of weather patterns, the interesting views are unknown, so exploration should be supported with a steering technique such as pointing.

Walking or redirected walking require large tracked areas, and physical locomotion devices can have serious usability issues. However, for some applications, especially training, where the physical motion and exertion of the user is an integral part of the task, such a device is required. First, however, consider whether your application might make do with something simpler, like walking in place.

Example: A sports training application will be effective only if the user physically exerts himself, so a locomotion device should be used.

Depending on the application, environment, and user goals, particular types of travel are likely to be common in a specific system, while others will only be used infrequently. The default navigation mode or controls should focus on the most common tasks.

Example: In a desktop 3D game in an indoor environment, most travel will be parallel to the floor, and users very rarely need to roll the camera. Therefore, a navigation technique that uses left and right mouse movement for camera yaw, up and down movement for camera pitch, and arrow keys to move the viewpoint forward, backward, left, and right would be appropriate for this application.

Simply providing a smooth path from one location to another will increase the user’s spatial knowledge and keep her oriented to the environment (Bowman et al. 1997). The movement can be at high speed to help avoid cybersickness. This approach complements the use of wayfinding aids. Teleportation should only be used in cases where knowledge of the surrounding environment is not important.

Example: Unstructured environments, such as undeveloped terrain or information visualizations, can be quite confusing. Give users spatial context by animating their target-based travel with a brief, high-speed motion.

If spatial orientation is especially important in an application, there are some simple strategies that will help users obtain spatial knowledge (Bowman, Davis et al. 1999). These include flying up to get a bird’s-eye view of the environment, traversing the environment in a structured fashion (see section 8.9.1), retracing paths to see the same part of the environment from the opposite perspective, and stopping to look around during travel. Users can easily be trained to perform these strategies in unfamiliar environments.

Example: Training soldiers on a virtual mockup of an actual battlefield location should result in increased spatial knowledge of that location. If the soldiers are trained to use specific spatial orientation strategies, their spatial knowledge in the physical location should improve.

You-are-here (YAH) maps combine a map with a YAH-marker. Such a marker helps the user to gain spatial awareness by providing her viewpoint position and/or orientation dynamically on the map. This means that the marker needs to be continuously updated to help the user match her egocentric viewpoint with the exocentric one of the map.

The most common solutions to this problem are less than satisfactory. Walking in place is simply not realistic or controllable enough for the game we envision. Redirected walking isn’t possible in tracked areas the size of most people’s living rooms. Applying a scale factor to walking so that the physical space maps directly to the virtual space doesn’t make sense when the virtual space is so much larger (and when it contains multiple floors), and it’s also difficult to use for fine-grained maneuvering tasks. Teleportation, while popular in many VR games, can be jarring and disorienting. Players have to plan for where to teleport so that they will be able to move away from a physical boundary (wall) once they arrive, which is cognitively demanding and reminds players of the real world, breaking presence. So the typical behavior with teleportation is to always stand in the center of the space and teleport every time you need to move even a little bit.

Let’s break the problem down a bit. First, consider navigating within a single virtual room. Assume that the game requires a minimum play area of 2x2 meters but that virtual rooms can be bigger than that. To allow players to physically walk around the entire virtual room, we could use many of the travel techniques described in the chapter, all of which come with tradeoffs. In general, purely virtual techniques would cause users to become reliant on virtual travel, so that they rarely perform physical walking. Scaled walking techniques could work within a single room, but could lead to sickness and lack of control.

In the end, we decided on a modified form of teleportation that still encourages (actually, requires) physical walking. We will design our virtual rooms to be composed of multiple cells, with each cell being the size of the tracked play area. Obstacles on the floor (chasms, debris, etc.) will indicate that the user can’t walk beyond the border of the cell. To get to a different cell, users will gaze at a special object in the cell and perform some trigger action (a gesture or button press) that will cause a rapid movement to that cell.

The key is that the movement will start at the current location in the current cell and end at the corresponding location in the target cell. In this way, the mapping of the physical play area to the cell boundaries is still correct, and users can reach the entire cell by physically walking around it. Since users don’t get to choose the exact target location of the movement, they have to walk within the cell to get access to the virtual objects therein.

How does this technique fit into the story of the game? Many narratives are possible. For example, we could say that you have a bird as a companion, and when you raise your hand the bird comes and picks you up and takes you to the part of the room you’re looking at. This would explain how you get past the barriers and also why you’re not in complete control of where you end up.

So what about moving from room to room? Since doors will by definition be near the boundaries of the physical space, moving through a virtual door will put players in a position where they can’t move any further into the new room without running into the wall. Again, we could use teleportation to move from the center of one room to the center of an adjoining room, but we think a more creative solution that could actually enhance the story is possible.

Imagine that all the doors in our fictional hotel have been sealed shut by the bad guys. But the designers of the hotel, envisioning just such a scenario, included some secret passages that we can use. These take the form of those fake bookshelves you’ve seen in dozens of bad spy flicks, where pressing a hidden lever causes the bookcase to spin around into the room on the other side of the wall. So to go through a wall, players have to stand on a semicircular platform next to the bookshelf and activate it somehow. Then the bookshelf and platform rotates them into the room on the other side. In this way, the user-room relationship is correct, and the user can physically walk back into the play area (Figure 8.23). This should also be fun and engaging. Sound effects will also help.

When using the rotating bookshelf technique, we want to avoid making the user feel sick due to visual-vestibular mismatch. We suggest providing a clear view not only of the current room but also the room behind the bookshelf. For example, we could cut a couple of holes through the bookshelf so users can see the room beyond before they activate the technique. If the user knows where she’s about to go and sees the rotation through the holes (with the bookshelf as a fixed frame of reference), that should reduce feelings of sickness.

The last piece we have to consider is vertical navigation from one floor to the next. In a play area with only a flat floor, it would be very difficult to do believable stairs, although there are some interesting techniques for climbing virtual ladders (Lai et al. 2015; see section 8.8.3). In a hotel environment, however, virtual elevators are the obvious choice.

Like all of the designs we’ve presented in our case study, this is only one of many possible ways to interact effectively in our VR game. But it does provide some unique ways to navigate the large environment of this virtual hotel in a way that should seem natural and fluid, despite the challenges of a limited physical play area.

Key Concepts

The key concepts that we used for travel in the virtual reality gaming case study were:

Natural physical movements for navigation can enhance the sense of presence.

Even with a limited tracking area, consider ways to allow and encourage the use of a physical walking metaphor.

If the application allows, use story elements to help users make sense of travel techniques.

We tried solving these limitations by developing a multiview camera navigation system. In other words, we deployed multiple cameras to allow users to take different viewpoints at the site, rather than limiting them to the view from their own physical location. The system included the cameras of the handheld units, but also a pan-tilt unit and several cameras mounted underneath a large blimp. Our system not only enabled the user to view the site from different perspectives to gain a better awareness, but also to share viewpoints with other users at the site, supporting collaborative work.

Implementing the right techniques to support multiview navigation encompasses both cognitive (wayfinding) and performance issues. To advance spatial awareness, we had to develop techniques that enhance overview possibilities and deal with occlusion issues, while conveying correct spatial relations (Veas et al. 2012). Users had to clearly understand the spatial features of the site to be able to interpret the augmented information correctly. We assumed that multiple camera viewpoints could provide a better overview of the site that could aid in creating a more accurate mental map of the environment. This principle is well known from the area of surveillance systems, but our system approach differs from such systems. In surveillance systems, the cameras are mostly static, while the observer is expected to have good knowledge of the site and the location of the cameras around the site. In contrast, the HYDROSYS setup also consisted of dynamic cameras and thus dynamic viewpoints, which may not easily be processed by the user, as she may not know the site, and the information displayed in a camera image might be difficult to match to the mental map.

Processing of the spatial information into the mental map is affected by discrepancies between the camera view and the user’s own first-person view (Veas et al. 2010), including the camera position and orientation, and the visibility of objects and locations. Clearly, cameras that are not from or in the user’s point of view and that are looking at another part of the scene are more difficult to process mentally. The key to an effective navigation technique is the approach used for traversing between the local and remote views. Such an approach might provide additional data such as maps or 3D models that convey the spatial configuration of a site, but that information might not always be readily available, or it might be difficult to match map/model data to ad hoc views.

In an initial evaluation (Veas et al. 2010), we showed that users prefer a technique that imposes lower workload if it allows them to perform reasonably well. Still, the evaluation also showed room for improvement in our initial techniques.

In the final system, we took a hybrid AR and 3D model exploration approach (Veas et al. 2012). Users could assess the locations of the different cameras by either looking through the lens of the selected camera or by switching to a 3D exploration mode. In the latter, the user could freely explore the underlying 3D model of the site at hand, similar to exploring Google Maps in 3D. This offers the advantages of resolving occlusion more easily and providing a good overview, since users do not need to stick to the available camera viewpoints. However, the 3D model has the disadvantage of being a static and potentially outdated model of the environment.

The interface showed the camera viewpoints (Figure 8.24) with regularly updated thumbnails of their video footage, and a mini-map could also be displayed. The user could switch to another camera’s viewpoint by selecting it directly or in a menu.

We also experimented with a method called variable perspective. This mode is an AR visualization technique (Figure 8.25) developed to combine views from different perspectives in a single image. It seamlessly blends between two different cameras: the first-person point of view and a second camera that can be rotated to visually “flip” the scene at further distances. This effect is similar to a famous scene from the movie Inception, in which the world is bent towards the user. The environment simply “curls upwards” at the switch point between first- and third-person perspective, thus presenting a gradual transition between both viewpoints. If the third-person view comes from a bird’s-eye view, this helps users see parts of the site that might be occluded from the first-person view.

Key Concepts

The key lessons that we learned from the mobile AR case study for travel were:

Situation awareness: creating a good mental map of the observed environment is crucial to adequately making use of the augmented information within. The acquisition and processing of spatial knowledge at larger-scale sites can be an issue, particularly if users have not visited the site before. The 3D UI needs to provide users with an overview, using techniques such as mini-maps and multi-camera setups.

Multiview: the use of multi-camera systems can help by providing an overview and resolving occlusions. However, we need to carefully consider how users switch between viewpoints, as switching can result in confusion. Combining AR and 3D exploration modes can be beneficial for providing contextual information about the spatial relationships between the cameras.