In a binaural context, sound source separation has been applied to speech in hearing-aid systems for the reduction of the cocktail party effect. The same is true for mobile telephony, where the reduction of environmental noise plays a critical role in the intelligibility of speech. However, even the listening experience of music, both at home and in concert halls, can be enhanced for the listener, as interfering environmental noise sources can be eliminated or attenuated.
In a broader single-channel or multi-channel context there are other compelling reasons for desiring to separate sound sources. Traditionally, the listener is considered as a receptor who passively listens to the audio signal stored on various medias (CD, DVD audio, etc.) or streamed through the internet. The only modifications that are easy to perform are global to the whole piece, like changing the volume, the tone or adding artificial reverberation. Although new formats such as MPEG Audio Layer 3 (MP3) have changed the way people access music, the interaction with music is still very limited. However, with the availability of higher computing capabilities, people are more eager to interact with the original media, while the sound is playing. This can be seen, for example, with the karaoke, where the listener can replace the voice of the original singer. But more freedom and creativity are also possible.
With the techniques presented in this chapter, new ways are available for the identification, separation and manipulation of the several sound entities (sources) which are perceived by the listener as independent components within the binaural (stereophonic) mix that reaches his/her ears. More precisely, one can find out the sound entities by considering their localization (spatial hearing) and the correlations of their spectral parameters (common onsets, harmonic relations, similar time evolutions). Then, it is possible to apply digital audio effects to each separate sound entity.
This way, listeners are enabled towards an active listening behavior, which entails freedom to interact with the sound in real time during its diffusion. For example, the listener can explore the possibility of changing the spatial locations of the individual sound sources, their relative volumes, pitches and even timbres, as well as their durations, or the rhythm of the music and experiment with these alterations while playing the piece. In other words, by means of sound source separation, the engaging world of mixing and re-editing separate tracks is reopened to the listener without the need to store or stream the separate tracks, which would require much larger storage or higher data rates.