3.1 Depth Cues and Applications to 3D Display

To perceive a 3D scene by a human visual system (HVS), the display system should be designed to present sufficient depth information for each object such that the HVS can reconstruct each object's 3D positions. The HVS recognizes an objects' depth in the real 3D world through depth cues, so the success of a 3D display depends on how well the depth cues are provided such that the HVS can observe a 3D scene. In general, depending on how many viewpoints are provided, the depth cues can be classified into two main categories: monocular and binocular.

3.1.1 Monocular Depth Cues

The category of monocular depth cue consists of depth cues observed from only one sequence of 2D images captured from one viewpoint. Depending on the availability of temporal information, it can be further classified into two subcategories: spatial monocular depth cue and spatial-temporal monocular depth cue. We list three major spatial monocular depth cues in [1]:

• Focus/defocus cues: The HVS associates the focus with a known depth plane, which is often refered to as accommodation. For an image captured by an aperture camera, the objects associated with the focusing plane of the optics are sharper than the other objects located away from this focusing plane. HVS can perceive an object's depth by differentiating the sharper objects from the blurred objects through accommodation.
• Geometric cues: Geometric cues include linear perspective, known size, relative size, texture gradient, and height in picture. The depth cue from linear perspective comes from the real-life knowledge that parallel lines intersect at infinite distance in a 3D world, but distances between parallel lines vary according to the related depth when they are shown in a 2D image. Similarly, the size information also reveals depth cues. When there are multiple objects with the same physical size shown in a 2D image, the closer object will look bigger. The gradient of a texture also exhibits the same depth cue: the gradient of an observed texture is more spread out when it is closer to the observer. The height in picture refers to our general experience that the objects shown at the bottom of an image is often closer than the objects shown nearer the top.
• Color and intensity cues: The color and light perceived by the eyes may vary according to the distance between object and eye. This category includes atmospheric scattering, light and shadow distribution, figure-ground perception, and local contrast. When light rays are scattered by the atmosphere, further objects have a bluish tint and with less contrast than the closer objects. HVS can utilize this real-world color experience of atmospheric scattering depth cue to tell the depth. The shadow cast by the sun or other light sources will vary according to the depth of the object in a 3D scene. The figure-ground perception refers to the perceived depth from edges and regions. When two regions are separated by an edge, one region at this edge has a definite shape (called the figure) and the other region appears shapeless (called the ground). The depth cue is perceived as the figure looking closer than the ground.

Spatial-temporal monocular depth cues include occlusions and motion parallax. This kind of depth cue is obtained when the viewer watches a sequence of 2D images along the time domain. The occlusion depth cue comes from the observations that background objects are covered by foreground objects, and the covered areas may increase or decrease when objects have movement. Motion parallax refers to the observations that when viewers move, the movement of further static objects is slower than the movement of the closer ones.

3.1.2 Binocular Depth Cues

When a viewer can perceive two images from two different viewpoints, the binocular depth cues may be observed. The binocular depth cues mainly include angular disparity and horizontal disparity [2]:

• Angular disparity: The angular disparity is the angle between two lines representing the viewing directions from that left and right eyes to the targeted object. It is also referred to as vergence angle.
• Horizontal disparity: When a particular point in a 3D world is captured on two images from two horizontally separated viewpoints, the projected points on the left image plane and the right image plane may not have the same horizontal position in each image's coordinates. The distance between these two projected points is called the horizontal disparity.

When the angular disparity or the horizontal disparity is zero, the viewer cannot observe the depth.

From the above discussions, a 3D display system should contain sufficient depth cues such that the viewers can observe the depth. The most effective approach for 3D display systems is to explore the binocular depth cues by showing the captured left image to the left eye and right image to the right eye. Based on this principle, there are different 3D display technologies that have been developed. Depending on how a stereo image pair is multiplexed from and/or de-multiplexed to left image and right image, the 3D display can be categorized as a stereoscopic display or an autostereoscopic display.