8.4 Binaural parameter synthesis

8.4.1 Mono down-mix

The synthesis process comprises reinstating the binaural parameters on a mono down-mix signal x(t) of the object signals. Using a frequency-domain representation, one frame of the down-mix signal is given by:

The power in each band of this down-mix signal frame, p_x, is then given by:

The reconstructed binaural signals Ŷ_t Ŷ_r are obtained using a matrix operation W_b that is derived for each parameter band (b) independently:

with D(.) a decorrelator which generates a signal that has virtually the same temporal and spectral envelopes as its input, but is independent of its input. The matrix coefficients ensure that for each frame, the two binaural output signals Ŷ_t Ŷ_r have the desired levels, IPD and IC relations. The solution for W_b is given by (see Chapter 5 for a detailed explanation of these equations):

with

8.4.2 Extension towards stereo down-mixes

In the previous sections, binaural parameters were analyzed and reinstated from a mono down-mix signal X(f). For several applications, however, it is beneficial to provide means to extend the down-mix channel configuration to stereo. An example of a relevant application scenario is the synthesis of a virtual multi-channel ‘home cinema setup’ based on a conventional stereo down-mix signal, as will be outlined in more detail in Section 8.5. A popular down-mix matrix equation for down-mixing five-channel material to stereo is given by:

with Y_l,ITU, Y_r,ITU the stereo down-mix signal pair, and X_lf, X_rf, X_c, X_ls, X_rs the signals for left-front, right-front, center, left-surround, right-surround, respectively, and q = 1/. Hence signals from loudspeakers positioned at the left side are only present in the left down-mix signal, and signals from loudspeakers positioned at the right side are only present in the right down-mix signal. The center channel is distributed equally over both down-mix channels.

The corresponding binaural signals Y_t, Y_r using HRTFs are given by:

When comparing Equations (8.35 and 8.36), two important differences can be observed. Firstly, the ITU down mix is frequency independent, while the HRTF matrix comprises frequency-dependent transfer functions. Secondly, the zero-valued cross terms in Equation (8.35) are replaced by nonzero HRTFs. If one assumes that the coherence of the virtual center channel (which is equal to the coherence between HRTFs H_l,c and H_r,c) to be (sufficiently close to) +1, it can be shown that the coherence of the binaural signal pair Y_l, Y_r resulting from Equation (8.36) will be equal or higher than the coherence of the ITU-downmix signal pair Y_l,ITU, Y_r,ITU. This observation has important consequences for a system that reinstates the binaural parameters of a certain binaural signal pair Y_l, Y_r based on an ITU down-mix Y_l,ITU, Y_r,ITU, namely that no decorrelator is required in the synthesis process. Instead, it is possible to derive a 2× 2 matrix (sub-band-wise, in a similar way as described in Section 8.4) that converts the ITU down-mix to a binaural signal Ŷ_l, Ŷ_r of which the binaural parameters are equal to those of the signal pair Ŷ_l, Ŷ_r resulting from convolution of the original input signals with HRTFs (assuming the HRTF parameter stationarity constraint as outlined in Section 8.3):

The matrix entries of W_b now follow from two separate binaural parameter estimation processes. The first column of W_b represents the binaural parameters stemming from all the original input signals (or virtual sources) present in down-mix signal Y_l,ITU (hence, X_lf, X_c, and X_ls), while the second column of W_b represents the binaural properties resulting from virtual sources X_rf, X_c, and X_rs:

with IPD_L the net IPD resulting from the combined virtual sources X_lf, X_c, and X_ls, IPD_R the net IPD resulting from X_rf, X_c, and X_rs, λ₁₁ the amplitude ratio between Y_l and Y_l,ITU, λ₂₁ the amplitude ratio between Y_r and Y_l,ITU, λ₁₂ the amplitude ratio between Y_l and Y_r,ITU, and λ₂₂ the amplitude ratio between Y_r and Y_r,ITU.