8.5 Multiple Description Coding

Multiple description coding is a source compression technique where the bit stream at the output of the encoder instead of having the usual single coded representation of the source, it now has multiple representations. For example, Figure 8.12 illustrates the particular case of a dual description encoder and decoder, where the number of descriptions is two. Encoding using multiple description coding also has the property that during decoding, each description can be decoded independently of the others, each resulting in a reconstruction of the source that can be considered to have a baseline quality. At the same time, the decoder can combine multiple descriptions (in principle, those that had been received with no errors in a communication setting) and obtain a reconstruction of the source with better quality than the baseline obtained from individual descriptions. Multiple description codecs are usually used in communication scenarios where each description can be communicated through a different link, independent of the others. In this way, each description will be affected through channel impairments that are independent of the others. With multiple description coding, receiving only one description is all that is required to recover a description of the source of usually fair quality. If more than one description is received, they can be combined at the decoder to obtain a reconstruction of the source with better quality. It is because of this operation that multiple description coding is often used for transmission diversity schemes at the physical layer. These transmission diversity schemes achieve through different means transmission paths that are independently affected by channel impairments.

Multiple description (MD) codecs were first studied in the context of rate distortion theory. The rate distortion performance of MD codecs was first studied by Ozarov, [13]. Succeeding works studied the use of MD codecs for communications over parallel channels [14, 15]. Also, as mentioned earlier in this chapter, MD coding can be straightforwardly applied in error resilience schemes where the bit stream at the output of the source encoder contains embedded redundancy. In this case, the particular approach seen with MD coding aims at making it possible to recover at least a representation of the source with basic quality. This relation between MD coding and error resilience has resulted in many research works studying this application, such as those in [16, 17].

Within MD codecs, the dual description ones are of particular interest because their simpler configuration has allowed for better understanding of its theoretical performance limits, measured through the rate-distortion (RD) function. In principle, there is no limitation to prevent each description being encoded at a different rate. Therefore, let R_D1 and R_D2 be the source encoding rates for the first and second description, respectively. At the receiver, if either of the two descriptions is decoded independently of the other, the achievable DR function follows the same performance as for single description coding:

Figure 8.12 A dual description encoder and decoder.

When the two descriptions are combined and decoded together, the achievable DR function is equal to:

in the low distortion case defined when D_D1 − D_D2 − D_M < 1 and equal to:

in the high distortion case.

Of all the possible sources, video, be it single or multi-view, offers perhaps the largest and best example of the use of multiple description coding. For example, a simple implementation of a dual description 2D video codec can split the input video stream into odd and even numbered frames, obtaining two video sequences with half the frame rate as the input sequence. Then, the encoder compresses each of the two video sequences independently of the other, generating two coded descriptions. At the receiver, if only one of the descriptions is received unaffected by channel impairments, the decoder can still output a reconstruction of the original video sequence of lower quality because either the output video will be of half frame rate or it would have the original frame rate but with the missing frames being estimated through interpolation of the frames in the single recovered description. If, instead, both descriptions can be decoded at the receiver, it will be possible to recover a video sequence with the same quality as the original. Nevertheless, note that the compression efficiency of this dual description video codec will not be as good as the one for an equivalent single description video codec. This is because the motion estimation and differential encoding of the half frame rate sequence is not as efficient as that for the full rate sequence.

Multi-view and 3D video present more opportunities for multiple description coding than those found in 2D video. Two possible multiple description 3D video coding schemes are presented in [18]. The first, illustrated in Figure 8.13, is a direct extension of the even-odd frame split described above for 2D video. The only difference in this case, besides the obvious one of now having two views, is that the descriptions are fully complementary of each other. That is, while description 1 encodes the even-numbered frames, with an I-frame on the right view and disparity prediction encoding on the left view, description 2 has the I-frame on the left view and disparity prediction on the right view.

The second multiple description 3D video coding scheme presented in [18] is illustrated in Figure 8.14. This scheme includes more sophisticated processing than the previous one. As shown in Figure 8.14, both descriptions encode all frames. While description 1 encodes the left view using disparity prediction, description 2 encodes the right view using disparity prediction. Furthermore, in each description the frames of the views being encoded using disparity prediction are also downsampled. The operation of downsampling a video frame consists in performing a two-dimensional low-pass filtering, followed by decimation of the frame. The decimation is implemented by keeping only every other pixel in the frame being decimated. The end result is that the downsampled frame has a lower, or coarser, spatial resolution. Usually, with each downsampling pass, the spatial resolution is halved. In [18], the implemented downsampling factors were two and four, meaning that the implementation of the downsampling followed one and two passes, respectively. For this scheme, the results presented in [18] show that the quality when only being able to decode a single description is still acceptable. The penalty in extra bits used during encoding depends on the level of encoded video sequence disparity correlation. When the disparity correlation is small, the number of extra bits used in this dual description scheme is also small and, of course, the number of extra bits used in this dual description scheme is large when the disparity correlation is large.

Figure 8.13 Frame interdependencies for a dual description 3D video codec with descriptions based on odd-even frame separation.

Figure 8.14 Frame interdependencies for a dual description 3D video codec based on downsampling the disparity-predicted frames in each description.

Some of the 3D video codecs used in the schemes earlier in this chapter result in compressed video bit streams containing two components: color information and depth information. A multiple description coding scheme based on this type of codec is presented in [19]. The approach taken can be seen as a combination of the different techniques already discussed in this section. First, a dual description codec can be implemented by separating the odd and even frames into two descriptions, each with their corresponding color and depth components. Next, in order to reduce the extra bits added by the embedded redundancy in multiple descriptions, downsampling can be applied to the color and depth components of the descriptions. Here there are multiple possible options and combinations onto which component to apply decimation. For example, decimation could be applied to only the depth components of both descriptions or it could be applied to the depth components of both descriptions and to the color information in alternate frames from the descriptions. The downsampled component can be transmitted to reduce the number of transmitted frames or it is also possible to introduce another form of multiple description by including in the coded bit stream both the non-downsampled and the downsampled component. Because for 3D video reconstruction quality purposes, color information is more important than depth information, it is preferable to transmit an extra description of downsampled color information before doing so for depth information. As a matter of fact, in [19] a good number of different combinations are tried. The most complete case is where, for each of the two descriptions, even and odd numbered frames, both downsampled and non-downsampled depth and color components are transmitted. A less complete, but more efficient, configuration is where for each of the two descriptions, even and odd numbered frames, both the downsampled and non-downsampled depth and only the downsampled color components are transmitted. A variation of this configuration is where, for each of the two descriptions, even and odd numbered frames, both the down-sampled and non-downsampled depth and only the non-downsampled color components are transmitted. The performance results discussed in [19] for these schemes show that when the channel impairment consists of relatively low packet loss rate, the overhead in extra transmitted bits due to multiple descriptions is a performance penalty that results in a loss of quality of approximately 2 dB. But, when channel impairments are in the form of a relatively large packet loss rate, the multiple descriptions provide better end-to-end quality.