Chapter 8
A Shot at the Assessment of 3D Technologies
As an assessment of 3D technologies is unavoidably associated with the future and as nothing is known to be more risky to predict than the future, attempts at this could be called audacious. However, contrary to predictions in the area of humanities, technological predictions can be based on already established laws and measurements from which – and that is the still risky portion – trends that appear promising can not only be detected, but also evoke enlightening discussions in a most beneficial way.
It may be best to approach this assessment very cautiously by first staying with 2D images and investigate physiological means of deriving 3D sensations in the form of illusions from planar 2D images. This enhancement of 3D perception will have to be realized electronically by additions to the addressing circuits.
We investigate this approach first for still 2D images.
A first set of depth cues in 2D paintings has been used by artists since the medieval period. Among them are the following.
Painting a road leading into the distance with a vanishing point and with objects decreasing in size along the way into deeper depths, as shown in the Figures 4.25 and 4.26, with bright and warm colors like white, yellow, and red in the foreground and more subdued and colder colors such as blue in the background. Occlusions and shadows cast away from the viewer also support the sensation of depth, as demonstrated in Figure 6.8a,b.
As elaborated upon in the first part of Section 2.5 and in Sections 4.62 and 4.63, luminance, contrast, and sharpness that are all larger in the foreground and become smaller toward the background are proven to enhance depth perception. A further powerful means of enforcing the sensation of depth is to increase luminance, gray scale, and contrast from the top of an image down to the bottom, as demonstrated in Figure 4.26a. This gray-scale conversion and dynamic contrast enhancement should be most pronounced in the foreground and become less so in the background.
An object pointing toward the viewer and narrowing to almost a point in the direction of the viewer while becoming brighter can evoke the impression of protruding from the image plane.
The aspects related to luminance, contrast, sharpness, and color can be controlled by the addressing voltages across the pixels.
The first task is to determine the depth in a real-time process, for which we focus on only three depths, namely near, middle, and far.
For depth-related luminance we sort the incoming pixel voltages into a series of three groups: high for near, medium for middle, and low for far. As the incoming voltages are numbered according to the row-wise series of locations of the pixels, the numbering reveals the area that the pixels belong to.
For contrast, the largest difference of voltages in a selected neighborhood is the main criterion, which is determined also with high for near, medium for middle, and low for far.
For sharpness the gradient of the voltages is the criterion. The majority in the three criteria determine to which depth group the pixel areas are finally assigned.
The enhancement of the depth cues requires changes in the value of the voltages according to their depth category. In the near category luminance, contrast, and sharpness have to be increased by increasing the voltages, their differences, and their gradients. Further, in all categories, luminance, contrast, and sharpness in addition have to be enhanced from the top down to the bottom.
Images with movement offer the possibility of enhancing depth by tracking the motion vector and assigning to the two eyes images from different times of the motion. As these two images are not applicable in our present 2D case, this excellent depth enhancement is only feasible when dealing with 3D images.
The enhancement of depth for still 2D images could be implemented by a chip performing the sorting and real-time changes of the voltages, which is attached to the existing addressing circuit.
The first commercial realization of 3D TV was stereoscopic followed by autostereoscopic displays. Common to both is their use of two images for 3D perception.
Stereoscopic displays require glasses, but the viewer is not fixed to a given location for 3D perception. Two types of glasses will coexist for some time, namely, the heavier shutter glasses, which require a battery to turn the LCD on and off, and the lighter weight passive glasses which reply on circular polarization. Linear polarization has the drawback of exhibiting a degrading sensitivity to tilting of the head. Active retarders with circular polarization are among the most promising approaches.
The two images are presented either in the spatial multiplex mode where only half the resolution of an FPD screen can be realized, or in the time sequential mode which allows for full resolution but requires double the speed of the addressing circuit. This speed is also necessary for addressing 240 Hz frames in order to achieve a virtually blur- and crosstalk-free 3D presentation. The higher speed can be handled by parallel addressing as shown in Figures 2.24 and 2.25 and by the recently introduced interleaved addressing as in Section 2.7. The combination of the two provides an attractive effect in that the image rate is still 240 Hz yet the addressing speed could be halved corresponding to a rate of 120 Hz. This time sequential system merits close attention for TV applications even though it requires two addressing circuits and slightly increased power consumption.
For autostereoscopic displays the simplest solution uses a light guide with two light sources for the two images and a 3D film realizing several (so far three) viewing positions, as presented in Section 3.6. It lends itself to 3D display for mobile devices; however, its extension to TV should be investigated.
More complex solutions, but still fit for multiview 3D displays, as long as the viewer stays in a plane in front of the display, rely on lenticulars or barriers, both fixed or adjustable. The lenticulars are brighter and the barriers are more easily switchable between the 2D and 3D modes. For lenticulars and barriers, time sequential operation with its larger resolution seems to be a very attractive approach.
All autostereoscopic displays offer the appealing feature that a viewer in the viewing plane can look at an object from the side. For those perspectives to be realized the LCD has to provide side view information at the pertinent viewing angle. This is achieved by the special arrangement of pixel information as depicted in Figures 3.9, 3.10, 3.12, 3.18 and 3.20. A consequence is that the LCD has to offer a much larger volume of information than for a 2D image. This is a severe and not yet satisfactorily solved problem for all 3D displays. One could visualize a solution by LCOS, which allows for a much higher pixel density.
This perspective view is not possible in stereoscopic displays where the viewer always perceives the same image from all viewing angles.
The next three 3D technologies, namely, integral imaging, holography, and volumetric displays, are the only approaches able to display a true 3D image. This means that the viewer is offered a perspective of the objects from different viewing angles without being confined to special viewing positions.
The capability for a perspective view from confined viewing positions offered by autostereoscopic displays with lenticulars or barriers is based on a similar optical structure as the pickup stage in Figure 5.1 for integral imaging. So laboratories familiar with lenticulars should have ready access to integral imaging (II). At an exhibition in May 2011 a perspective view realized by II could already be admired. So one might assume that II could be the first true 3D technology to make it to the consumer market even though this event is still years away.
The first step in the creation of an II image by a pickup stage with lenticulars does not fit into the present capture of an image by a camera. So this is a first hurdle to be overcome on the way to commercialization. The second step with the reconstruction stage in Figure 5.3 could be visualized as being implemented on a currently used FPD. However, basic features such as the enhancement of depth, viewing angle, and resolution still require more attention. The most advanced II structure seems to be a projection system in which the lenticulars are able to realize simultaneously both the real image and the virtual image. These images are adopted to present two views of a scene from different depths as shown in Figures 5.22 and 5.23. This is a convincing approach for true depth reality. The pickup plate with the elemental images for the projector could be prepared by a non-real-time fabrication process. This would allow movies to be presented but not real-time TV.
Holography is scientifically the most appealing approach to true 3D perception. Similar to II, the preparation of the hologram, the equivalent of the pickup plate, is a non-real-time process. Digital computer-generated holograms, made by a non-real-time process, are a very successful means of presenting 3D images in medicine, microscopy, and other mainly scientific applications. For electrical engineers and physicists working in optics, the description of a picture element by phasors, as presented in Section 6.2, establishes easy access to the understanding of complex reference waves, real and virtual holograms, as well as offset reference and detour phase holograms. The exact or approximate real-time realization of a fast Fourier Transform (FFT) is an essential achievement necessary for the wider use of 3D holography. Ample knowledge of this topic available from digital signal processing could prove very stimulating.
Volumetric 3D displays suffer from the experience that simple solutions with only two or three stacked displays, each carrying the image from a different depth, do not provide satisfactory depth perception. Therefore attempts to create satisfactory images with these small numbers of displays finally had to revert to adding an autostereoscopic component, that is, an image for the right and the left eye as presented in Sections 3.5 and 7.2. A well-accepted volumetric solution consisted of 20 stacked displays which rendered it suitable mainly for professional applications. It might be worthwhile to further explore the benefits and drawbacks of the two- or three-stack approach combined with the two eyes' view.
Finally, attention is drawn to the very attractive work of computer scientists on the assessment of quality of 3D images and on the understanding of cues, including the depth cue, in images as reported in Chapter 4. Cooperation between the computer group and the designers of 3D TV systems has already started and could become very fruitful in the future. This is especially true for the area of depth image base rendering (DIBR), treated in Section 4.7, where the transmission of the 3D color HDTV signals for the two eyes, with a limited bandwidth and still with enhanced depth cues, is also discussed. In this context depth perception can be enhanced according to Pulfrich's phenomenon by using motion parallax for the two eyes' view. This is done even though this parallax is not associated at all with depth. The assignment of a depth has so far been done in a heuristic way, as described in Section 4.5, and should be understood more precisely.