In the context of this book, we are looking for representations of sound signals and signal-processing systems that can give us ways to design sound transformations for a variety of musical applications and contexts. It should have been clear throughout the book that several points of view have to be considered, including a mathematical, and thus objective, perspective, and a perceptual, and thus mainly subjective, standpoint. Both points of view are necessary to fully understand the concept of sound effects and to be able to use the described techniques in practical situations.
The mathematical and signal-processing points of view are straightforward to present, which does not mean easy, since the language of the equations and of flow diagrams is suitable for them. However, the top-down implications are much harder to express due to the huge number of variables involved and to the inherent perceptual subjectivity of the music-making process. This is clearly one of the main challenges of the book and the main reason for its existence.
The use of a spectral representation of a sound yields a perspective that is sometimes closer to the one used in a sound-engineering approach. By understanding the basic concepts of frequency-domain analysis, we are able to acquire the tools to use a large number of effects processors and to understand many types of sound-transformation systems. Moreover, as frequency-domain analysis is a somewhat similar process to the one performed by the human hearing system, it yields fairly intuitive intermediate representations.
The basic idea of spectral processing is that we can analyze a sound to obtain alternative frequency-domain representations, which can then be transformed and inverted to produce new sounds (see Figure 10.1). Most of the approaches start by developing an analysis/synthesis system from which the input sound is reconstructed without any perceptual loss of sound quality. The techniques described in Chapter 7 are clear examples of this approach. Then the main issues are what is the intermediate representation and what parameters are available for applying the desired transformations.
Figure 10.1 Block diagram of a simple spectral-processing framework.
Perceptual or musical concepts such as timbre or pitch are clearly related to the spectral characteristics of a sound. Even some common processes for sound effects are better explained using a frequency-domain representation. We usually think on the frequency axis when we talk about equalizing, filtering, pitch shifting, harmonizing… In fact, some of them are specific to this signal-processing approach and do not have an immediate counterpart on the time domain. On the other hand, most (but not all) of the sound effects presented in this book can be implemented in the frequency domain.
Another issue is whether or not this approach is the most efficient, or practical, for a given application. The process of transforming a time-domain signal into a frequency-domain representation is, by itself, not an immediate step. Some parameters are difficult to adjust and force us to make several compromises. Some settings, such as the size of the analysis window, have little or nothing to do with the high-level approach we intend to favor, and require the user to have a basic signal-processing understanding.
In that sense, when we talk about higher-level spectral processing we are thinking of an intermediate analysis step in which relevant features are extracted, or computed, from the spectrum. These relevant features should be much closer to a musical or high-level approach. We can then process the features themselves, as shown in Figure 10.2, or even apply transformations that keep some of the features unchanged. For example, we can extract the fundamental frequency and the spectral shape from a sound and then modify the fundamental frequency without affecting the shape of the spectrum.
Figure 10.2 Block diagram of a higher-level spectral-processing framework.
Assuming the fact that there is no single representation and processing system optimal for everything, our approach will be to present a set of complementary spectral models that can be combined to cover the largest possible set of sounds and musical applications.
Having set the basis of the various spectral models, we will then give the details of the implementation techniques used both for their analysis and synthesis processes, providing MATLAB® code to implement a complete analysis-synthesis framework.
In the final section we will present a set of basic audio effects and their implementation based on the analysis-synthesis framework just introduced. MATLAB code is provided for all of them.