9.3 Two-View Gaming Systems

Video games need reality and exciting components to attract customers. Therefore, new technologies are developed and easily accepted by the video game community. Since people see the world with their own two eyes, 3D stereoscopic techniques can provide an extra sense of presence. These 3D techniques become new attraction points for the video game industry. Furthermore, stereoscopy gives developers the ease of picking up things that look fake. Depth is important for correct perception of the virtual world, and setting the depth of the virtual scene properly is critical to the success of a game. The stereoscopic development environment can provide an extra tool for developers to check and correct visual defects by giving proper depth cues in the game before they are even used in a game during the development procedure. According to our discussion in Chapter 2, 3D polygonal meshes are the fundamental representations for games. As shown in Figure 9.5, applications deliver stereoscopic content to the customers by the following steps:

1. Applications issue geometry-based rendering commands to the graphics card.
2. The graphics card generates the frames for the left and right views through a stereoscopic pipeline which can be one of two possible forms as shown in Figure 9.6.
3. The two views are sent to the 3D vision-ready display units and the 3D-ready display delivers the interlaced images to the left eye and right eye separately.
4. Pairs of stereoscopic glasses help customers extract each view separately and the customer's brain can reconstruct the 3D perception of the scene from these views.

There are mainly two possible rendering pipelines for generating stereoscopic views as shown in Figure 9.6.

1. Rendering the scene twice

   The polygonal meshes of the 3D world are fed into the original graphics pipeline with two camera settings to represent the left view and right view respectively. The scene must run through the entire pipeline twice to render frames for the left view and right view. Therefore, the rendering process requires twice the amount of rendering time when compared to rendering a monocular view for a single 3D stereoscopic view and the resulting frame rate is approximately half of the original one.

   

   Figure 9.5 The 3D game content delivery procedure.

   

   Figure 9.6 (a) The process of generating two views by running the traditional pipeline twice for each view respectively. (b) The process of generating two views by running the traditional pipeline once for the left view and DIBR once for the right view.

2. Rendering once with DIBR

   A depth map is the byproduct of a traditional graphics pipeline because the z-buffer algorithm is the default visibility algorithm. Therefore, depth-image-based rendering (DIBR) can be added after the original rendering to generate the stereoscopic view of the scene at little cost. There are two extra advantages for this category. As discussed in Chapter 2, DIBR also decouples the rendering process and the displaying process to give players the ability to adjust their preferred disparity setting. The extra DIBR process can be naturally extended for free-view gaming systems.

NVidia, which is one of main manufacturers and developers of graphics processing units (GPUs), saw the commercial possibility of 3D stereoscopic technologies and started to develop 3D stereoscopic hardware and software for new 3D applications. The company revealed the NVidia 3D Vision Pro to incorporate with their manufactured graphics cards as solutions for delivering the highest quality and the most advanced 3D stereoscopic environment. The 3D environment enables designers, engineers, digital artists, and scientists to see and share their work in true 3D. As shown in Figure 9.7, there are four components needed:

1. Graphics processing units (GPUs)

   A GPU is the core technology, which provides realism of the virtual world to players in modern games technology. The 3D stereoscopic games require even more computational resources from the graphics hardware and thus a proper GPU is necessary for 3D stereoscopic games. For example, the newest version of NVidia GeForce, Quadro and Tesla GPUs can be used.

2. 3D vision-ready display

   The rendering results must be displayed on a device for the player to see and interact. Currently, glass-less autostereoscopic monitors are only commercially available on mobile devices and not for LCD monitors, televisions, or projectors. Therefore, the commercially available 3D display devices generally require a pair of glasses to watch 3D content. Pairs of active shutter glasses provide better control and visual quality and thus, pairs of active shutter glasses should be the main choice for 3D stereoscopic games. Therefore, the fundamental requirement for 3D vision-ready displays is that the refresh rate must be over 120 Hz to prevent the lag problem of temporally multiplexed 3D-glass display devices.

   

   Figure 9.7 The devices and software required for 3D gaming.

3. Stereoscopic glasses

   3D movies can be seen through a pair of red–cyan glasses or polarized glasses because of the price, but the artifacts are generally more serious than with a pair of active shutter glasses. Therefore, it is recommended to use pairs of active shutter glasses for achieving better stereoscopic perception.

4. Stereoscopic drivers

   Stereoscopic 3D games require programmers to set up the viewing conditions for each eye in every frame. The setting procedure is cumbersome. A good 3D driver should provide an intuitive and automatic control to set the 3D stereoscopic viewing conditions.

Setting proper depth of the world is one of the important tasks for designing a 3D stereoscopic video game. Therefore, here are several issues related to depth setting which must be considered before developing a 3D stereoscopic game.

1. Depth setting represents the maximum stereo separation.
2. Stereo separation is the distance of a given object as shown in the left view image and the right view image.
   • Maximum stereo separation should be equal to the distance between the two pupils of a person which is roughly about 2.5 inches.
   • Some people may feel uncomfortable with a large amount of stereo separation and thus the default value is generally set to 15% of the maximum stereo separation.
   • Most users will raise it to a higher value after their eyes become used to the 3D stereo contents.
   • If the stereo separation of an object is equal to 0, the object appears at the screen depth.
   • Distant objects have separation quickly approaching maximum, which is a function of 1/D where D is the depth of the object.
3. Generally, the 3D viewing system should provide a convergence control to set the depth for the zero stereo separation, screen_depth.

In addition, the developer also would like to maximize the 3D stereoscopic experience of players when playing games. The following are several hints to enhance the experience:

1. Place objects at their correct depths.
   • Since our eyes use multiple cues to determine the distance from the eye to an object, the rendering of objects must be consistent in these depth cues to deliver the right depth perception.
   • Occlusion is only a weak depth cue but it generally provides the relative distance to the viewer. If object A is obscured by object B, it is a strong hint that object A is further away.
   • When stereo separation contradicts with other depth cues, eyestrain and headache will result.

     For example, the skybox is a common technique used in a game. A skybox is generally rendered at the beginning of the rendering process without using the z-buffer visibility algorithm. Therefore, some random D value which is unrelated to the rest of the scene is assigned to the drawing. This D value may induce issues in depth perception of the scene. Therefore, when rendering skybox or other far-away background objects, their D values must be set to a value which is larger than the D value of the rest of objects in the scene.

2. Maximum stereo separation is about 2.5 inches which is roughly 100–200 pixels in a common game setting. Stereo separation is a function of 1/D and can quickly approach the maximum value.
3. According to the experience, the D range between screen_depth and screen_depth × 100 can generate a vivid 3D effect. Therefore, try to fit as many scene objects within this range as possible.
4. Set screen_depth to the maximum possible value (using “convergence adjustment”) to enhance the 3D effect. However, the process must make sure that objects are inside the screen and do not come out of the screen and get clipped by the screen sides.