2

3D Graphics and Rendering

With the advance of technology, stereoscopic 3DTV is now available and popular in the current electronics market. The fundamental concept of a 3DTV system is to deliver stereoscopic 3D perception of the world to viewers while watching streams of 2D videos. These 2D videos are sent to each eye separately and later the brain fuses them together to generate the perception of the world. In other words, 3DTV delivers the content of a 3D scene to consumers' 3D displayable devices for generating 3D perception to viewers. The 3D content may be virtual (synthesized by computers) or realistic (captured from real scenes) and can be transmitted in various ways and forms according to the types of the scene, the level of required realism, the type of applications, the available bandwidth of the transmission channel, and so on. The representations of the 3D scenes is the bridging technology among the procedures of scene extraction, modeling, transmission, and viewing for a 3DTV framework. However, the requirements of scene representations in each procedure are often very different and even sometimes conflicting (e.g., transmission rate versus visual quality). To achieve these different requirements, different 3D scene representation techniques are available and generally categorized into three fundamental classes according to whether geometric models are used or not:

1. Rendering with explicit geometry: The first category is rendering with explicit geometry which has direct 3D information. It is also called a geometry-based representation and can be roughly divided into two different categories:

   (a) Surface-based representations: This category, which includes polygon meshes, nonuniform rational B-spline (NURBS), subdivision and point-based modeling, is popular due to its generality, extensive support by current hardware, and enriched interactivity. However, the ability to provide realistic rendering is the largest limitation to meet the requirement of 3DTV.

   (b) Volume-based representations: They have the ability to encapsulate neighborhood information which is important for parallel processing the reconstruction of surfaces for multi-view algorithms. However, the largest issue is the trade-offs between resolution and memory usage or transmission bandwidth.

   More details will be given in Section 2.2

2. Rendering with no geometry: These representations use no geometry information but only 2D images or videos to represent the 3D scene and are also called image-based or texture-based representations. The representations encode the different views in a single image or video or in a set of images or videos. Without geometry, the sampling must be very dense, or the possible motion is restricted. Examples in this category include [1–4]. The main advantage is the inherent realism because the represented images and videos are captured from the real world. In other words it can easily give viewers the sense of realism. However, the memory and/or the transmission rate of this type of representation is a serious issue. In addition, the provided interactivity has its own limitation. More details will be discussed in Section 2.3 and Chapter 5.
3. Rendering with implicit geometry: This category relies on positional correspondences across a small number of images to render new views. The methods are also called depth-image-based representations. The term implicit implies that the geometry is not directly available. The representatives include [5–9]. Depth-image-based methods can be viewed as a hybrid of the geometry-based and image-based representations in order to gain the advantages of both categories and alleviate the disadvantages of both categories. Depth-image-based representations are appropriate for free-view 3DTV applications, since they can generate the virtual views easily. They are currently the popular choice for stereoscopic representations. More details will be discussed in Section 2.4 and Chapter 5.

Figure 2.1 is a diagram which shows the representation spectrum, ranging from one extreme, geometry-based representations, to the other, image-based representations. In this chapter, we provide a short description of these important techniques and approaches which could be used for 3D scene representations. In addition, we also provide a short discussion about the pros and cons of each method.

Figure 2.1 This shows the categorization of scene representations for 3D applications and 3DTV systems.