10.3 Peer-to-Peer 3D Video Streaming

The server–client streaming paradigm has its limitation owning to the bandwidth bottleneck and is not scalable to support a large group of end users. The problem will become worse for the multi-view video streaming service. It has been demonstrated that the peer-to-peer (P2P) video streaming can significantly alleviate the computation load and bandwidth requirement experienced in a centralized video streaming system. Basically, the P2P streaming system can be categorized into two different approaches: tree-based approach and mesh-based approach, as illustrated in Figure 10.7.

Figure 10.7 Peer-to-peer streaming approaches (a) tree-based, (b) mesh-based.

The tree-based approach will construct a structured network in a tree topology. The server will serve as the root of the tree and each peer will play both the role of the parent and child node. The video delivery path is built by starting from the server and relaying the streams by peers through the connected nodes, which can provide a low latency video delivery, and thus is suitable for video broadcasting. The advantage of this approach is a shorter join time for a new peer since there is no peer discovery phase needed during the initialization. However, the peer leaving will bring extra handling for the tree reorganization, especially for a peer serving as a parent node who needs to link its parent and child node together to avoid leaving its child out of the tree. In addition, since each parent node is responsible for its child node, the required outbound bandwidth for each peer has higher requirement.

The mesh-based approach is to build an unstructured network where each peer connects to multiple peers. The video delivery path is set up in both ways between each pair of peers, and therefore the concept of parent node and child node is blurred in this network. The mesh-based approach requires extra time for peer discovery when a peer wants to join the streaming service. Therefore, the target application for mesh-based approach is the video-on-demand service. The requirement of the outbound bandwidth is not as high as the tree-based since each peer only needs to contribute a portion of the video data for its neighboring peer. Because the connectivity of mesh-based approach is higher than the tree-based solution, the mesh-based approach will suffer fewer problems when a peer chooses to quit the current video service.

Since the multi-view video streaming requires high bandwidth consumption, deploying a multi-view video service in a distributed and cooperative fashion is expected to alleviate these problems. A simple solution for multi-view video over P2P networks can be constructed by encoding each view independently (i.e., no inter-view prediction) and transmit each view along each independent P2P streaming delivery path. Owing to the distributed nature of P2P networks, users will experience a longer waiting delay for view synchronization by leaving the current video delivery path completely and requesting to join a new video delivery path when the view switching is requested. A proper design of joint/quit protocol and peer management is needed to reduce the waiting latency [22–24].

One way to alleviate the long view-switching latency is to construct the two-layer overlay architecture, which takes advantage of the MVC coding structure and consists of cross-view overlay and intra-view overlay [25]. As illustrated in Figure 10.8, the cross-view overlay delivers the global I-frames as they are needed for every view and the intra-view overlay transmits both the required B/P-frames in each view and the referenced B/P-frames from neighboring views. In other words, users in the same cross-view overlay may watch different views and users in the same intra-view overlay watch the same view. Each user should join these two overlay trees. During the normal playback within one view, each user receives the required video bit streams for the required I/P/B-frames from these two trees. When one user wants to switch from view 1 and view 2, the node will send notification to both overlay neighbors. Some intra-view neighbors who contribute the most content to the user will become this node's new neighbor in the cross-view overlay. The cross-view neighbors who play back view 2 become this node's new intra-view neighbors and send the local and referenced B/P-frames for view 2 to this user. By doing so, the view-switching latency can be reduced.

The multi-view bit streams may experience bit corruption or packet lost during transmission. Multiple description coding (MDC) is already shown as an effective solution to overcome those transmission problems. A video program encoded in MDC format consists of multiple descriptions, and each description can be decoded independently and rendered to provide a basic visual quality. The more descriptions the decoder successfully receives and decodes, the better final visual quality the end user will observe. Therefore, MDC streams are useful in a distributed environment to provide progressive quality refinement. Bringing MDC technology into a P2P network, each end user will have the basic video quality with receiving at least one description from his/her peer. When the degree of connectivity from other peers increases or inbound channel bandwidth becomes larger, the end user can receive more descriptions for higher visual quality. This is different from the traditional single description method in which the end user waits for all segments from all peers to arrive to start a smooth playback, especially for real-time streaming scenario. Further improvement on the final reconstructed 3D video quality via MDC is expected when the bit streams are transmitted through a lossy channel condition [26].

Figure 10.8 Two-layer overlay architecture (a) intra-view overlay, (b) cross-view overlay.

Precise synchronization for left view and right view also is a key factor to determine the success of a 3D P2P streaming service. To achieve the best 3D viewing experience, the left view and the right view video frames should be available for rendering at the targeted time instant after the playback starts. Having one view missing will downgrade the 3D viewing to 2D viewing; and having two views missing will result in playback jitter. An intuitive way to achieve perfect synchronization is to pack the left and right views that have the same display timestamp together in the same P2P segment. However, this will increase the required bandwidth between peers and cannot fully utilize the distributed manner in P2P streaming. A proper design for receiving each view separately from different peers can improve the playback performance. The scheduler can be further formulated as an optimization problem and solved via binary quadratic programming [27].