i
i
i
i
i
i
i
i
4.2. Technology for Interaction 65
video output to two projectors. The graphics processors are fast enough to run
the distortion-correction software without the need for any special hardware.
4.2 Technology for Interaction
Once the basic decision has been made about the type of display technology
that will be utilized for the virtual environment, y ou also have to decide how
to include interaction with the other senses:
• The sound system is relatively easy to prov ide and control. A number
of software libraries, for example Microsoft’s DirectSound [13], allow
application programs to easily render 3D soundscapes which match the
graphics.
• We will see in Chapter 10 that the sense of stereopsis adds greatly to
the perception of reality. The commonly used method of providing this
effect is by using infrared (IR) controlled shutter eyewear. This tech-
nology requires the IR signal to be received by the eyewear at all times,
and so the cave designer must install a sufficient number of emitters to
allow all the users of the cave to pick up the signal.
• Head-mounted display VR systems must employ motion tracking.
Without it, the correct viewpoint and orientation cannot be gener-
ated. In a cave, it might at first sight not seem so important to know
where the observers are. Nevertheless, the global component (the cave)
and the personal component (the head-mounted display) ar e likely to
merge in the future and a g ood cave designer should recognize this and
include some position-sensing technology. One way to do this is by
including an ultrasonic emitter into the roof and walls of the cave, as
described in Section 4.3.
• In the introduction to this chapter, we discussed the importance of user
interaction through input or combined input/output devices. Within
the desktop, you have the luxury of working within a relatively small
working volume. In a cave, this is not the case, and as such, natural
interaction is much more difficult to achieve.
We feel that touch is next most important sense (after vision) for a user
to engage with whilst within the virtual environment. As such, the remaining
i
i
i
i
i
i
i
i
66 4. Building a Practical VR System
sections will describe the current state of the art in input devices which allow
us to manipulate objects within the virtual environment and output devices
that allow us to sense the reaction to any changes we made. That is, they
stimulate our sense of touch. These are commonly referred to as haptic devices,
and before we describe both the input and haptic output devices, we thought
we might give you a crash course in the theory of haptics.
4.2.1 Haptics
In Section 2.3, we discover ed that the human haptic system has an impor-
tant role to play in human interaction with VR. But we have also seen that
whilst haptic technology promises much, it comes with many unresolved
problems and complexities. State of the art haptic interfaces are still rather
crude.
However, it is a rapidly developing topic with new ideas emerging regu-
larly from research labs. Srinivasan and Basdagon [18] and Salisbury
et al. [17] provide interesting general details of how haptic systems are clas-
sified and the challenges of trying to build haptic hardware inter faces. In
designing a VR system, and especially a cave-type system, we would ideally
like our haptic device to be able to work at long range, not get in the way,
and offer the ability to appear to pick up a virtual object and get a sense of its
weight. By extrapolation, if we could do that, most of what we would want
to do with haptics would be possible.
Following [18], a useful way to describe a haptic device is in the way it is
attached to a fixed location.
• Floating devices are things such as gloves which can perform inter-digit
tasks. They can also measure finger contacts and finger-specific resis-
tance. They cannot however measure or reproduce absolute weight or
the inertial effects of a virtual object. They may only be attached to
their base location via wireless link.
• Exoskeleton devices, typically worn on a hand, arm or leg, may have
motorized devices that can resist certain motions and restrict the num-
ber of degrees of freedom. For example, such a device may prevent you
from closing your hand too tightly around a virtual ball as you attempt
to pick it up. This type of haptic device, like the first, does not allow
you to experience a weight effect.
i
i
i
i
i
i
i
i
4.2. Technology for Interaction 67
• Grounded devices behave rather like the arm of a small robot. They can
be powerful and simulate multi-axial forces, including everything from
a solid wall to picking up a virtual object and assessing its weight. They
have the disadvantage of being bulky and very restricted in the volume
in which they can move.
In the real world, we can make quite a range of movements, termed de-
grees of freedom (DOF). In three dimensions, six DOF provides a complete
specification of position and orientation. As such, a haptic device need only
specify a force along three mutually perpendicular axes, but that force could
depend on position and angle at the point of contact. Many haptic devices,
such as the force-feedback joysticks used in computer games, can only simu-
late a force along two axes and at a single point of contact (since the joystick
does not move).
An ideal haptic device has to have some formidable properties: low inertia
and minimal friction, with device kinematics organized so that free motion
can be simulated and have the ability to simulate inertia, friction and stiffness.
These devices can achieve tactile stimulation in a number of different ways:
• Mechanical springs activated by solenoid, piezoelectric crystal and sha-
pe memory alloy technologies.
• Vibrations from voice coils.
• Pressure from pneumatic systems.
• Heat-pump systems.
H owever, these are all still technologies under active research, and most haptic
feedback is offered via joystick or data gloves. In addition, a haptic device is
obviously useless if it does not have haptic-enabled VR software controlling
it. When developing this type of software, it is helpful to imagine haptics
operating in an analogous way to the 3D graphics with which we are familiar.
The term haptic rendering says it all. One specifies a haptic surface; it could
consist of the same shapes and even be specified in the same way with planar
polygons that the 3D renderer uses to visualize the shapes for us. Continuing
with the analogy, we can texture the surface by describing its rigidity, friction
etc. just as we do in terms of color, reflectivity etc. for the visuals.
Figure 4.14 is a block outline of the structure of the haptic-rendering
algorithm. The gray box shows the main elements in the haptic-rendering
loop. The device sends information about the position of its point of contact.
i
i
i
i
i
i
i
i
68 4. Building a Practical VR System
Figure 4.14. Haptic rendering.
The collision-detection block uses information from the simulation software
about the shape of the haptic and visu al surfaces to determine whether the
virtual position of the haptic device has bumped into the felt surface. The
force-r esponse block determines an appropriate force depending on the state
of the collision (hard, soft, deformable etc). It sends this signal to the block
that controls the electronics of the haptic device to reproduce this force and
also to the simulation engine so that changes to the shape of the felt surface
can be made if necessary.
In Chapter 17, we shall look at how an application program might im-
plement the details of these blocks. But now we will quickly summarize what
each block has to do:
• The collision-detection block does exactly what its name implies. It de-
tects when and where the virtual representation of the real world point
of contact collides with the rendered haptic surface. It may also reveal
other information that helps us simulate deformable objects such as the
depth of penetration through the haptic surface.
• The force-response block calculates the force that approximates as closely
as possible the real force that the avatar would experience in the
collision. As stipulated in Chapter 3, an avatar is a computer gen-
erated graphical representation of the human user within the virtual
environment.
• The control section translates the force felt by the avatar into signals
to drive the haptic hardware. Because haptic devices are limited in the
force they can deliver, it may not be possible to exactly simulate the
force one would feel if the object had been real and we had bumped
into it.
i
i
i
i
i
i
i
i
4.2. Technology for Interaction 69
• In the simulation block, the effect of the haptic exchange is applied to
the data that describes the haptic surface and possibly the visualization
data, which might be represented by the same geometry.
Undoubtedly, haptics is going to be a major focus for R&D in the
future. Most indications suggest that a lot of current research is focused
on small-scale haptic topics, for example tactile perception of deformable ob-
jects and simulation of realistic haptic textures. There seems little prospect
of large-scale haptic rendering such as we would like to have in a cave envi-
ronment. It seems that some kind of a breakthrough is needed, because we
cannot clutter up the real working space with a collection of robot arms, such
as would be needed to really get to grips with objects in the parallel virtual
universe.
After this brief aside, we now return to consider the input and haptic
output devices used for both desktop and immersive VR wor k.
4.2.2 Desktop Interaction
In a desktop VR environment, the user may only require some basic interac-
tion with the VR world, such as 3D navigation. Devices such as a trackball
or a 3D joystick can offer this facility. Other forms of input such as light
pens and drawing tablets may be only be marginally relevant for VR work.
Figure 4.15 illustrates examples of some of these devices.
Figure 4.15. A wide range of desktop devices extend the range of input possibilities. A
spaceball (a) offers the possibility to move in three dimensions. The joystick is a low-
cost device with two or three degrees of freedom. Some degree of haptic feedback
can be achieved with basic joysticks (b). To give a comprehensive haptic response
requires a device such a s SensAble’s Phantom (c).
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.