Sound design is ongoing throughout the production process—in preproduction planning, during production, and in postproduction. In one respect it is a continuous process that “covers the spectrum from music on one end to dialogue at the other end with sound effects in between. The key to integrating it all is that the concept of sound design is to make sure these various elements work together and don’t step on each other’s toes.”[1] But in another respect, it is more than plugging in dialogue, sound effects, and music. Sound design is the vision that harmonizes the various elements with the visual story by creating the overall sonic character of a production.
In terms of film and video, sound design is to producing the sound track what production design is to producing the picture. The production designer is responsible for the overall look of a production; the sound designer is responsible for the overall sound—both, of course, with the director’s approval. Dialogue is entirely in the director’s hands.
Responsibility for a sound design falls to a designated sound designer, who advises the audio team and, ordinarily, has a hands-on role during one or more stages of the sound track’s production. The term sound designer was first used as a professional craft designation in theatrical film in 1979 when an academy award for sound design was given to Walter Murch for Apocalypse Now. Over the years the term has been diluted to include just about anyone on the audio production team, particularly creators of sound effects and sound-effect editors, who has a hand in producing the audio. Using the preferred meaning, the sound designer is the one person responsible for creative control of the audio. It is the sound designer’s function to put a coherent sonic stamp on the production—and the one member of the audio team who, like the production designer and the image editor, has the director’s ear.
Regardless of whether there is a designated sound designer, all members of the audio team contribute creatively to the sound in a production. Hence the material in this chapter applies to sound design in general and the various components of audio in particular rather than to a specific role carried out by an individual member of the audio production team.
Whatever an audio team member’s function (or anyone in sound production for that matter), there are two overriding qualifications: having the ability to listen perceptively and knowing how to effect what the audience is to perceive. Or, in the parlance of audio, having “ears.”
For the audio professional, one fundamental requirement is having educated ears; that is, the ability to listen with careful discrimination to style, interpretation, nuance, and technical quality in evaluating the content, function, characteristics, and fidelity of a sound. (To have educated ears, however, it is essential to have healthy ears; see Chapter 1.)
Most day-to-day aural stimuli we take for granted if not actively tune out. We may pay attention to a particular song or follow a conversation, or some sound might catch our attention—an alarm, a barking dog, breaking glass—but most sound registers as little more than background to our comings and goings. To the person in audio, however, such a lack of awareness is a liability. An audio professional must be sensitive to all sound. Innate sensitivity to sound varies, and not everyone has the same perceptual acuity.
You can acquire certain skills, however, through training in audio and music courses. Eartraining resources also help (see Bibliography). Study the sounds of the environment—both natural and built. Listen to the soundscape with its myriad voices, textures, and rhythms. Go for a listening walk or a soundwalk, where you focus your attention on disciplined, active listening. Take advantage of the vast and rich resources from the audio worlds of film sound tracks, TV, radio, and music. When you don’t listen, sound remains part of the landscape—it does not become part of your consciousness.
Developing an ear involves discrimination, the ability to shut out all sounds but the one you are listening to, to hear one sound among many. This is known as “putting your ear to it.”
It can be argued that many sounds offer so little aesthetic satisfaction that they are not worth listening to. Many sounds even annoy, such as sirens, leaf blowers, blaring horns, music in an elevator, a loud TV from an adjacent apartment, and the sonic assault from sound systems in seemingly every public space—from airports to shopping malls to restaurants. It is enough to desensitize aural acuity and make it difficult to enjoy worthwhile sound. Hence it seems reasonable to conclude that listening should be selective. Such a conclusion might be appropriate for most people, but it does not apply to the audio professional.
Training the ear takes time. It comes with years of perceptive listening. You begin by learning how to listen, paying attention to sound wherever and whenever it occurs: in traffic, in conversation, at sporting events, at a concert or movie, or when dining, walking, showering, getting dressed, watching television, or just lying in bed. Remember, exposure to sound goes on during every waking hour and even during sleep.
You learn what to listen for by studying a sound both in context and by itself. That means acquiring the ability to listen analytically and critically. “Analytical listening is the evaluation of the content and function of the sound.... Critical listening is the evaluation of the characteristics of the sound itself [its technical integrity].”[2]
Words may denote meaning in speech, but sound often defines it. On paper the meaning of the phrase Have a nice day is clear, but the meaning changes when the stress on certain words changes or when the words are spoken with a lilt; in a monotone, whine, or drawl; or by an old man, a young woman, or a child. Sound in speech conveys such qualities as confidence, fear, anxiety, arrogance, humor, and concern. In an interview a person may appear confident, but if there are unnatural pauses between words or phrases, or if there is a parched quality to the speech, the person’s sound will belie the appearance. In dialogue a charming, good-looking character conveys a sense of menace when his words are delivered in a harsh and guttural voice. A character whose voice rises at the end of sentences conveys vulnerability, regardless of what is being said. Analytical listening is the evaluation of such sound, the interpretation of the nuances and the connotations of the sound quality in addition to—or in spite of—the words being spoken.
Take the sound of a dog barking. A bark is generally harsh and abrupt. But barks vary widely in pitch, loudness, rhythm, and context. For example, low-pitched barks last longer than high-pitched barks, and some barks begin with a gurgling-type sound rather than with a sharp attack. Within a bark may be a whine, yelp, growl, howl, or boom. Also some barks have regular rhythms, whereas others shift beats and produce irregular tempos. Each of these sounds tells you something about the dog and the situation.
The sound a chick makes while hatching may seem obvious: gradual cracking of the eggshell and then peeping. But listening to a hatching reveals more: the chick peeps inside the egg before cracking it; the peeping is muffled. The shell begins cracking slowly with short, tentative splitting sounds that increase in force. With the increase in force and slightly longer cracking sounds, peeping increases in clarity, loudness, and rapidity. The final cracks of the shell sound more like crunches as the chick stumbles into the world. Once the chick is out of the shell, the peeping is unmuffled, steady, and strong but not quite so loud as it was just before emergence.
In television, sound varies from program to program, even of the same genre. In sitcoms laughter may come from a live audience or a recorded laugh track. It may be more robust, or less, with a preponderance of elderly female or young male voices. In football there are different approaches to mixing the announcers’ sounds with crowd sounds (see Chapter 13). Some audio operators like to keep the level of crowd sounds and the announcers’ voices nearly the same to maintain excitement. Others prefer to keep the level of crowd noise relatively low so that, when the action justifies it, the level can be increased to accent the excitement.
Music presents the greatest challenge in listening. Its sonic combinations are infinite, and its aesthetic value fulfills basic human needs. Musical taste is intensely personal; two people listening to the same music may respond in two different ways, both valid.
Take two recordings of, say, Carl Orff’s Carmina Burana. Keep constant as many factors as possible, such as the quality of the digital discs, the recording format (stereo, surround sound, digital), the audio system, and the room. You may be surprised at the differences in sound and interpretation. You may prefer the sound in one recording and the interpretation in another. That does not mean that one recording is necessarily better; it means that, based on your perceptions, one is preferable to another for certain reasons. Someone else may disagree.
Do not listen to music based only on taste and mood. Go beyond recreational listening. Disengage your preferences; explore all types of music, whether or not you like and enjoy it.
In listening to those two recordings of Orff’s Carmina Burana, you may prefer the sound of one recording to the other because of its technical quality: the clarity of detail, the inflection of a particular range of frequencies, the balance of orchestral instruments and chorus, or the spatial imaging. Or in listening to a song, you may decide that it is darker sounding than it should be because the bass drum is muddying the mix, there is a buildup of too many frequencies in the low midrange, or there is an absence of color in the treble range.
Critical listening is the evaluation of the characteristics of the sound itself: when the vocalist enters a bit early or late to counterpoint the lyric against the accompaniment; when a sound effect has too much wheeze and not enough whine; music underscoring that fades up at the wrong time, masking dialogue; the polyrhythms that play off one another to enhance the rhythmic excitement of an action scene; the annoying surges in crowd sound during pauses in a sportscaster’s game call because the ratio and/or threshold settings on a compressor are set too severely; the sonic differences between an acoustic bass and an electric one or acoustic drums and a drum machine; the unique sound of a Zildjian cymbal or a Telecaster electric guitar; a concert grand piano with a percussive tonality or a tonality that “sings”; the musicians’ technical proficiency; the acoustics that enrich sound or dull it; stylistic differences among artists; or the sonic difference between analog and digital sound, between 16-bit and 24-bit mastered CDs, between 48 kHz and 96 kHz sampling rates, and between a CD and a DVD-Audio.
Critical ears hear differences in sound between microphones, changes in tone color from different microphone placements, coloration of a microphone preamplifier, the difference between +3 dB at 1,000 Hz and +6 dB at 3,000 Hz, between preamplifier overload and loudspeaker distortion, subtle phase cancellation, and the right frequency or frequency range to fix an equalization problem.
This is not to overwhelm you with detail or suggest that serious listening is entirely clinical. Feeling the sound and sensing what works are also critical to active analysis as is being aware of the relationship between your taste and mood and your response to sound.
Reading reviews also provides insight into why critics think a work does or doesn’t succeed. In fact, it is equally instructive to know why a recording or film sound track does not work as it is to know why it does, perhaps even more so.
Develop a sound vocabulary. Sound is nonverbal; it is difficult to describe. Coining words that are descriptive of the effect you are trying to achieve not only hones your own perceptions but also provides others on the sound team with an indication of what you are going for. There are hundreds of such terms used for sound effects, such as twung, squidge, whibble, wubba, boink, kabong, zuzz, and so on. In music there are terms such as honky, mellow, warm, bright, hollow, and zitsy (also see Chapter 22). Or, consider scientists who have come up with a variety of descriptors for the vocalizations of seals, including chi-chi-chi, chirrup, eeyoo, chug, what-chunk, chnk-chnk, too-loo, rrwhmp, and seitz.[3] Such terms are inexact, of course, and need objective association to make them clear to others, but they do have a sonic texture and, in some examples, a pointed “visual” impact by themselves.
Take an audio recorder with you wherever you go. Record sounds to study and to discover their sonic characteristics. Manipulate them in the studio for their potential to be processed into other sounds. Sound designers have been doing this for years so they do not miss the opportunity of recording new and unusual sounds to add to their sound-effect libraries. They bring their recorders to toy stores to take advantage of the rich array of sounds that toys generate; on vacations to document that unique sound of a wave that does not crash but laps against the shore; into bathrooms to capture the swirling, sucking sound of water whirlpooling down a bathtub drain; and around a neighborhood, marketplace, or public square to capture its particular sonic personality.
Because response to sound is personal, standards and guidelines are difficult to establish, so listening is the key to improving aural discrimination. The ear is capable of constant improvement in its ability to analyze sound. As your aural sensitivity improves, so will your level of auditory achievement. “Listening with the ear is inseparable from listening with the mind.”[4]
Creating a sound design involves sound effects (covered in Chapter 15) and music underscoring (Chapter 16). Paradoxically, two possible additional domains include silence (Chapter15) and the ability of sound to evoke a picture in the mind’s eye. This may not seem to be an impressive arsenal, especially when compared with the number of elements available in pictorial design—color, light and shadow, composition, scenery, costumes, physical dimension, focal length, camera angle, depth of field, and so on. Even though these aural elements are few, they are powerful agents in creating, supplementing, and complementing cognitive and affective information.
All sound—speech, sound effects, and music—is made up of the same basic elements: pitch, loudness, timbre, tempo, rhythm, attack, duration, and decay. During audio production everyone involved is dealing with these elements, consciously or subconsciously, and assessing their effects on perception. Because each element contains certain characteristics that affect our response to sound, understanding those effects is fundamental to sound design.
Pitch refers to the highness or lowness of a sound. High-pitched sound often suggests something delicate, bright, or elevated; low-pitched sound may indicate something sinister, strong, or peaceful.
Loudness describes the relative volume of sound—how loud or soft it is. Loud sound can suggest closeness, strength, or importance; soft sound may convey distance, weakness, or tranquility.
Timbre is the characteristic tonal quality of a sound. It not only identifies a sound source—reedy, brassy, tympanic—but also sonic qualities such as rich, thin, edgy, or metallic. Reedy tonal qualities produced by a clarinet or an oboe, for example, can suggest something wistful, lonely, or sweet. A brassy sound can imply something cold, harsh, fierce, bitter, forceful, martial, or big. A tympanic or percussive sound can convey drama, significance, or power.
Tempo refers to the speed of a sound. Fast tempos agitate, excite, or accelerate; slow tempos may suggest monotony, dignity, or control.
Rhythm relates to a sonic time pattern. It may be simple, constant, complex, or changing. A simple rhythm can convey deliberateness, regularity, or lack of complication. A constant rhythm can imply dullness, depression, or uniformity. Rhythmic complexity suggests intricacy or elaborateness. Changing rhythms can create a sense of uncertainty, vigor, or the erratic.
Attack—the way a sound begins—can be hard, soft, crisp, or gradual. Hard or crisp attacks can suggest sharpness, excitement, or danger. Soft or gradual attacks can imply something gentle, muted, or blasé.
Duration refers to how long a sound lasts. Sound short in duration can convey restlessness, nervousness, or excitation; more sustained sounds can create a sense of peace, persistence, or fatigue.
Decay—how fast a sound fades from a certain loudness—can be quick, gradual, or slow. Quick decays can create a sense of confinement, closeness, or definiteness; slow decays can convey distance, smoothness, or uncertainty.
Other aspects of sound, such as changing pitch, changing loudness, and acoustic interactions, also affect response. Of course, these elements are not heard individually but in combination.
Someone speaking in a loud, high-pitched voice at a rapid rate conveys excitement, whatever meaning the words may have. Lowering pitch, reducing loudness, and slowing tempo may also convey excitement, but this combination of sounds suggests something more profound and deeply felt. Words spoken at a deliberate tempo in a reverberant—that is, acoustically live—room may suggest a weightier content than those same words spoken in an acoustically drier environment.
These same factors can be applied to music and sounds. A trumpet or violin played loudly and rapidly at a high pitch could also suggest excitement, agitation, or gaiety—perhaps agitation in drier acoustics and gaiety in livelier acoustics.
Striking a wooden door or a metal barrel loudly and rapidly can also suggest agitation or gaiety. Again, lowering pitch, reducing loudness, and slowing tempo change the response to something more serious, whether the sound source is a trumpet, violin, wooden door, or metal barrel.
That these characteristics are elemental in sonic structure is not to suggest that sound design is prescriptive or developed by applying formulas; the basic components of sound structure do not occur separately but together in myriad forms. Rather, it is to introduce and define the building blocks of sound from which the sound designer shapes aural structure and meaning.
Sound designer Randy Thom commented that, for the filmmaker, “Sound, musical and otherwise, has value when it is part of a continuum, when it changes over time, has dynamics, and resonates with other sound and with other sensory experiences.”[5] In thinking about the character of a sound, the key is to ask yourself: Is it interesting? Is it engaging? Does it do any good?
For more than four decades, the sounds of radio drama created pictures in the audience’s “theater of the mind” (a characterization coined by poet Stephen Vincent Benét). Understood in the context of synesthesia, this is not so unusual a reaction. Synesthesia is the phenomenon whereby a sensory stimulus applied in one modality causes a response in another. For example, hearing a particular sound creates a distinct mental image or seeing a certain color mentally triggers a specific sound. (There are other synesthetic reactions not relevant to this discussion, such as a certain smell stimulating the mental image of a particular color, or a certain sound stimulating a specific taste.) For our purpose the synesthetic phenomenon demonstrates that the ear sees and the eye hears.
The implications of the power of sound to stimulate visual images is another formidable consideration in sound design. Among other things, it allows expanding the screen’s dimensions. Sound is not bound by screen size as picture is. Off-screen sounds can evoke additional images to what is seen on-screen, such as atmosphere, environment, and action, especially with surround sound. Even on-screen sound can add greater visual dimension to a shot. The on-screen shot shows a man sitting in a chair being questioned, but in the off-screen audio heavy footsteps shuffle by the interrogation room, people are heard screaming, and a volley of distant gunshots rings out (see also “Strategies in Designing Sound” later in this chapter).
When picture is present, the sound/picture relationship creates certain dynamics that affect overall meaning. As essential as sound is, in relation to picture its “great power is conditional. It places the image in an emotional and physical context, helping us to decide how to take the image and how it integrates itself into everything else.”[6] The sound/picture relationship is not a contest of superiority. It is a symbiotic relationship; they are two different “organisms” that are mutually beneficial to the whole. In general, there are five basic relationships: sound parallels picture, sound defines picture, picture defines sound, sound and picture define effect, and sound counterpoints picture.
When sound parallels picture, neither the aural nor the visual element is dominant. This function of audio in relation to picture may be the best known because it is often misunderstood to be sound’s only function. In other words, what you see is what you hear. A character drops a glass, and you hear it. A jazz combo plays in a bar, and you hear the music. On-screen a raging fire is consuming a building, and in the sound track are the sounds of metal warping, pipes creaking, puffy explosive whomps, and the roaring crackle of the fire itself.
This sound/picture relationship should not be dismissed as unworthy just because it is familiar and expected; not all shots require sonic augmentation.
When sound defines picture, audio not only is dominant but also determines the point of view—the subjective meaning of the image(s). Take a scene in which prison guards are standing battle-ready in front of a cellblock. A sound track consisting of crashing, breaking, yelling sounds suggests a prison riot and casts the guards as “good guys,” as protectors of law and order. If the same scene were accompanied by the song “Freedom,” the music then casts the prisoners sympathetically and the guards as their oppressors. The same scene augmented by a dissonant, distorted rendering of “The Star-Spangled Banner” not only conveys the idea of the guards as oppressors but also adds an element of irony.
Consider a picture showing a man apologizing to a woman for his many wrongs. Her look is stern and unforgiving. The music underscoring the scene is gentle and sympathetic, indicating her true feelings. In the same scene, if the music were light and farcical, both his past wrongs and his pleading would be conveyed as not very serious. Change the music to dissonant and derisive sound, and the woman’s stern, unforgiving look becomes nasty and gloating.
Picture helps define sound by calling attention to particular actions or images. In a scene where a person walks down a city street with traffic sounds in the background, cutting to close-ups of the traffic increases the impact of the sounds. The sound of metal wheels on a horse-drawn wagon clattering down a cobblestone street is intensified all the more when the picture changes from a medium shot of the carriage to a close-up of one of the wheels.
When sound and picture define effect, the aural and visual elements are different yet complementary. Together they create an impact that neither could alone. Take the obvious example of a wave building and then breaking on a beach, accompanied by the swelling and crashing sounds. Separately, neither sound nor picture conveys the overall impact of the effect. Together they do. Neither sound nor picture is dominant, and although they are parallel they are reinforcing each other to produce a cumulative effect. Another example would be an elderly man sitting on a park bench, watching a mother playing with her child, lovers walking hand-in-hand, and teenagers playing touch football, accompanied by a music track that evokes a feeling first of pleasure, then of loneliness, and finally of futility. Sound and picture are complementary, with both contributing equally to an effect that would not be possible if one element were dominant or absent.
When sound counterpoints picture, both contain unrelated elements that together create an effect or a meaning not suggested by either sound or picture alone—a tertium quid. For example, the audio conveys the sounds of different groups of happy, laughing people, while the picture shows various shots of pollution. The audio/video counterpoint suggests an insensitive, self-indulgent society. Or consider the shot of an empty hospital bed, with the music evoking a sense of unexpected triumph. The audio/video counterpoint creates the idea that whoever had been in the bed is now, by some miracle, recovered. Change the music to sound spiritual and ascending, and the person who had been in the bed will have died and “gone to heaven.”
Sound influences how we react to picture. Try watching television or a film first with the sound off and then with the sound but without the picture. Usually, you will find that more information comes from the sound than from the images. Two different sound tracks for the same picture will produce two different meanings. In a very real sense, creating a sound design is often tantamount to defining a production’s conceptual and emotional intent.
There is no set procedure for designing sound. At the outset the most important thing to do is study the script and analyze the auditory requirements line by line to determine the overall sonic approach to various scenes, for an entire work, or both. When there is a director, of course it is necessary to consult. There is no right or wrong method in this type of creative decision-making. For the director or sound designer, or both, decisions are often made on the intuitive basis of what feels best—of what “works” in serving the aesthetic intent of the whole. This is not to suggest that sound design is simply “touchy-feely.” Reasoned bases should underlie determinations.
Script analysis involves two things: determining the overall sound design to various scenes or to an entire work and spotting—deciding on the placement of sound effects and music (see “Spotting” in Chapters 15 and 16).
In deciding on a sound design, the overriding questions are: How is the audience to think or feel about a particular story, scene, character, action, or environment? And from what point of view? Moreover, is the sound design to be carried out mainly in the sound effects, the music, or both?
Here are some examples:
In a story about a reclusive, paranoid, unsuccessful writer, his environment, where there should be various sounds of life and living things, has only a few sounds, and those are spare, disjointed, reverberant, and mechanical.
In a documentary about tango dance styles, tango music is woven throughout the scenes of the story—from a dance hall to street scenes in Buenos Aires.
In a story about the Amish, who eschew synthetic, manufactured products, backgrounds and ambiences are devoid of any sounds that would be generated by such items.
In a story about a completely self-involved person, a lack of background sounds wherever he is, regardless of the locale, underscores his psychic isolation.
In a story that takes place in the 1930s, using audio equipment from that time to produce authentic effects and music tracks sonically frames the story in that period.
In a story set in Elizabethan England, using musical instruments and styles from that period in the underscoring adds to the story’s authenticity.
In a scene showing a raging typhoon at sea, designing the sonic effect of a ship’s being lashed by howling winds, masts and rigging snapping and straining, waves crashing on deck, the vessel heaving and groaning intensifies the scene.
In a scene taking place on a neighborhood street on the hottest day of the year, the ambience is dense and the sounds of sparse traffic, people strolling, and children playing halfheartedly are muted and slowed slightly.
In a scene attempting to provide the emotion of being in a vast, empty desert that is sterile and lifeless and where there is little sound, making a sonic fabric from insectlike sounds, such as faint clicking and scratching noises, and grains of sand rubbing against one another conveys a palpable sense of place.[7]
In a scene about building a car, tying the rhythm of the machines to the rhythm of the music underscoring creates a unifying gestalt.
In a jail scene, to convey the hard, tough, cold world of the prison, use the sound of iron doors closing with a metallic scrape and slamming shut with a sharp report; the hard sound of heels on concrete when walking; the hollow, reverberant, oppressive ambience; and orders being barked in harsh monotones.
In tight shots of a couple arguing as their marriage is falling apart, the ambience (air) is taken out of the sound track to convey the claustrophobic relationship as its life is being squeezed out.
In a scene meant to convey the sensory assault of information overload and the inundation of communication devices, the sound design is produced with layers of audio that include primary and secondary dialogue coming from different locations in a room, with some people talking on cell phones, a nearby television blaring the news, an answering machine playing back a message to vote for a particular candidate, and a radio in the next room hawking a commercial.
In a social gathering of friends meant to depict the characters’ insincerity and emotional detachment from one another, their dialogue is reproduced more through a conversational hubbub than through understandable speech. Voices can be understood without words; it’s not what is being said but how the voices are saying it that communicates.
In a scene creating the point-of-view (POV) transitions of a sightless person, the sonic environment changes as the camera shots and angles change so that what is seen is not only heard but magnified in sharpness and detail.
In a drama meant to have the feel of a documentary, the sounds are kept focused and austere and do not include all the sounds that would normally be heard if the approach to the design were typical. This helps prevent distraction and focuses concentration on what is being said and done on-screen.
In a harrowing war scene, one character is used as a focal point. To create the POV of his taking psychological refuge from it all, he witnesses all the mayhem of the battle but hears the weaponry, explosions, and screams of the wounded muffled and at a distance. Another approach is to underscore the entire action with gentle, deliberate, melodic, harmonious music to create a counterpoint between picture and sound.
In a story about love and lust, passion and jealousy, tragedy and betrayal, involving characters whose mutual interest is opera, arias whose lyrics express those emotions and conditions are used for the underscoring. Such an approach to sound design can be taken with any genre of music so long as it is consistent with the story and the characterizations.
The following are two examples of how sound design can affect the meaning of the picture. For the first example, let’s use one simple shot:
FADE IN
INTERIOR. BEDROOM. NIGHT. A LITTLE GIRL IS LYING ON A BED LOOKING AT THE CEILING. A SHAFT OF MOONLIGHT COMES IN THROUGH AN OPEN WINDOW ILLUMINATING ONLY PART OF HER FACE. THE CAMERA SLOWLY ZOOMS IN.
Clearly, the script’s story and the director’s choices would govern the sound design for this shot. But the intent of this example is to demonstrate sound’s significant influence on picture. Consider the shot with the following sound patterns and how each pattern influences its meaning and feel.
The thrum of an air conditioner and the polyrhythmic din of insects
Gentle breaking of waves and a distant foghorn
The sounds of conversation and revelry coming from another room
The sounds of a man and a woman arguing in an adjacent room and a sudden crash
The whine of air raid sirens, the rumble of airplanes, and the dull thudding of distant bombs
The girl breathing, humming softly—almost imperceptibly—to herself
A single dog howling, building to a scattered chorus of baying hounds in the distance
The creaking of floorboards and the sound of a door latch being opened
The distant sounds of a playground with children laughing and singing
Street traffic, the occasional car horn, and the booming bass of music from a passing car
These examples have other thematic possibilities, of course. Instead of using organic sounds, nonorganic sounds processed electronically could create different textures, moods, and environments. Nonorganic sounds are less realistic and have an otherworldly detachment; they tend to make a sound track less emotional. Conversely, organic sounds are more realistic and tend to make a sound track more emotional.
Now consider what underscoring does to the meaning of each of the previous examples, with or without the sound effects. With music that is, say, celestial, mysterious, sinister, playful, comedic, romantic, melancholy, blissful, animated, and threatening, notice how the idea and the feel of each shot changes yet again.
For the second example, let’s use this short scene: Mike and Paula, a couple in their midtwenties, are on the third day of a camping trip; they are emerging from dense woods into a wide clearing.
FADE IN
1.
EXTERIOR, WOODS—LATE AFTERNOON ON A SUMMER DAY. MIKE AND PAULA HAVE BEEN BACKPACKING AND ARE EMERGING FROM DENSE UNDERBRUSH INTO A CLEARING.
2.
MIKE STOPS AT THE EDGE OF THE CLEARING TO LOOK AROUND.
MIKE: Wow! This looks like a perfect place to camp for the night.
3.
PAULA, FOLLOWING FROM THE UNDERBRUSH, STANDS NEXT TO MIKE.
PAULA: It sure is. [BEAT] Seems kind of unusual though, doesn’t it?
MIKE: What?
PAULA: To have such a large clearing with no trees or plants in the middle of such dense woods. We’ve been hiking for almost two days and have barely seen a trail. Now all of a sudden, this. I mean, look at how well manicured it is. Almost as if someone’s been caring for it.
4.
MIKE TURNS TO PAULA AND SMILES. HE WALKS FARTHER INTO THE CLEARING, REMOVES HIS BACKPACK, AND DROPS IT TO THE GROUND.
MIKE: Are you complaining? For the first time in two days we can pitch the tent and not have to sleep in the woods on dead leaves—to say nothing of the cozier advantages.
5.
PAULA WALKS TO MIKE AND THEY KISS. SUDDENLY, A HOWL SOUNDS IN THE DISTANCE.
MIKE: What was that?!
PAULA: I don’t know! Never heard anything like it before!
6.
THEY LOOK AROUND SLOWLY, STARING INTO THE WOODS.
PAULA: Brrr. [BEAT] I don’t know whether I’m frightened or just getting chilly.
MIKE: It’s the wind. Starting to blow up a bit. [LAUGHS NERVOUSLY] That’s all we need—wind, then thunder and lightning. The only thing missing is the dark castle on the hill.
7.
AS MIKE SAYS THE LAST LINE, THUNDER SOUNDS IN THE DISTANCE.
This scene is brief, ordinary, and uncomplicated, yet its many sound cues can be handled in a number of ways to enliven it and give it impact. Let’s explore a few possible approaches to the sound design to convey some idea of the creative possibilities.
The scene opens in woods with dense underbrush in the late afternoon of a summer day. Among the elements to consider are the sounds coming from the woods and the sounds of walking through woods and on underbrush—whether they will be natural and benign or unnatural and foreboding.
One function that sound performs is to create an unseen environment. Woods exude a close atmosphere, a thickness of air that should be present in the sound. Woods are filled with sounds in summer: birds twittering, insects buzzing, perhaps frogs croaking, and, at night, an owl hooting or crickets chirping. By adding such sounds to the audio track, it is possible to picture the woods and provide a sense of density without actually showing it.
In this case, walking through woods and the underbrush requires a few different sounds: clothes scraping against branches, the crunch and the crackle of stepping on twigs or dry leaves, two pairs of footsteps—one lighter than the other—and, perhaps, increased rates of breathing or panting. The weight, rhythm, and sharpness of the sounds can detail the amount of underbrush and the characters’ level of fatigue.
If the purpose of these effects is to provide a contextual sonic complement to the picture, they should sound natural. If they are to suggest the frightening experience to come, they should be unnatural in some way, taking on a narrative function.
Any number of narrative approaches might work. The animal sounds could be edged, harsh, or screechy. Their rhythms could be chattering and nervous. The buzzing could increase or decrease in loudness, as opposed to a more natural steady drone, to reflect a sense of impending attack. The animal sounds could take on a nonorganic, not-of-this world quality. Or there could be silence—perhaps the most unnatural and ominous sound effect of all.
The footsteps could sound bigger than life, setting the stage for the theme of Man, the intruder; or they could sound smaller to convey the characters’ vulnerability. The snap of the branches could sound more like a whip, the crackle and the crunch of the twigs and the leaves underfoot like bones breaking or squooshy like the crushing of an insect—sonic foretelling of events to come.
Emergence from the underbrush requires a change in ambience, footsteps, and animal sounds (assuming they were present during the previous scene). The change from the closed-in woods to the large expanse of the clearing would be accompanied by a more open-sounding ambience. The sound of the footsteps would have to reflect the difference between the dry leaves and the firmer but cushioned meadow. The animal sounds may not be so intense, underscoring the difference between being in the woods and in the clearing. They may also sound more distant to further emphasize the change.
When Mike removes his backpack, it calls for the scrape of the straps on his shirt, maybe a grunt of relief, a sigh of satisfaction, or a slight “ahh” of anticipated pleasure. There are a few possibilities for the sound of the backpack hitting the ground: a bigger-than-life thud to emphasize its weight; a metallic thud to give a sense of what is inside; a jangly crash to sharply punctuate the quieter, open ambience, to provide a sense of what is in the backpack, and perhaps to foretell the jarring events to come.
Sonically, the kiss could be used to set the stage for the first howl. Before the kiss, background sound could diminish to silence, then subtly become more intense, climaxing with the howl; or the background sound could change, again subtly, from natural to unnatural.
The howl suggests a number of possibilities, not only as a sound but also as a narrative expression. Of course, any determination about it has to be made within the context of the story line. Let’s assume the howl is from a creature—half man, half beast—who is looking for revenge after being severely wounded by hunters. The howl should convey these facts. It should sound like a wail to establish the pain but be guttural enough to project the idea of a dangerous animal. The wail could embody human characteristics and vacillate expressively to convey both a sense of pain and a cry for help. The guttural edge would also provide a counterpoint to the wail by sounding angry and menacing.
A nonorganic approach could be taken using electronically generated sounds. This approach creates an alien creature, perhaps changing the half-man part to something more fantastic. In any case, not only the condition but also the personality of the creature would be conveyed by the sound.
The wind sound could be designed to convey a number of moods and conditions: mysterious, gloomy, empty, gentle, stormy, cold, and so on. As its frequency increases, wind becomes more active and turbulent.
In this scenario Paula says “Brrr,” so the wind should be pitched high enough to convey chill; because it has just come up, it should have some low-end sound as well to convey mystery or danger. A gradually increasing shrillness or whistle would suggest the impending storm.
Thunder has several sonic characteristics: ripping attack; sharp, deep-throated crack; rolling, continuous rumble; and gradually increasing and decreasing loudness as it moves closer and then farther away. In this case, because the thunder sounds in the distance, the sound design calls for a low-throated, continuous rumble. As it comes closer with the storm, its other sonic characteristics would be added.
The wind and the thunder also could be generated electronically. This would give the environment an alien quality, particularly if the animal sounds in the woods were also synthesized to suggest the presence of some unworldly beast.
These are only a few possible approaches to the sound design of this brief scenario. Music provides even more alternatives to the informational and emotional content; the possibilities are infinite. On the other hand, an absence of music could enhance the sense of isolation and threat. The role of music in providing and enhancing meaning is covered in Chapter 16.
Another important factor in the sound design of this shot is how the audio cues are mixed. Mixing layers the sonic elements and establishes their balance and perspective, aspects of sound that are discussed at length in Chapters 21, 22, and 23.
The point of the previous examples was to demonstrate the power of sound to change the impact and the meaning of given content. The discussion assumed that the sound designer had the mandate to originate such changes. In reality this rarely happens because interpretation of material is up to the director. Even so, a sound designer still has creative leeway in affecting the overall outcome of program material.
The following examples are common types of assignments a sound designer is called on to handle. Because context and content of the audio in these examples are prescribed, it is up to the sound designer simply to produce it, always with the director’s final approval, of course. The challenge is to see how imaginative the sound designer can be and still remain true to the director’s vision.
Some audio materials are produced by one person who, in effect, is both the director and the sound designer. This is often the case in radio and with spot announcements—commercials, promos, and public service announcements (PSAs). With spot announcements, particularly commercials and promos, the “message” is often in the dynamics of the audio design and less in what is actually said, as in this illustration of a promo for an all-news station.
SFX: HARD PAN—EFFECT WITH DRONE
ANNOUNCER 1: Keeping the commitment...
MUSIC: STAGER INTO AMBIENT SOUND
ANNOUNCER 1: With detailed coverage, this is news radio 1060...
MUSIC: STAGER IN AND FADE UNDER ANNOUNCER
ANNOUNCER 1: (PAN LEFT TO RIGHT WITH RISING EFFECT) W... Z... Y... X.
MUSIC: DRAMATIC MUSIC BED IN AND UNDER
ANNOUNCER 1: For complete coverage of the day’s news, keep your ears planted here.
SFX: SPLAT
ANNOUNCER 2: (High-pass filter) Apparently, that’s what being planted sounds like.
SFX: DRUMBEAT SOUND OVER LAUGH, SEGUE TO...
MUSIC: PULSING BEAT THEN UNDER
ANNOUNCER 1: These days news is not funny. (PAN LEFT TO RIGHT WITH RISING EFFECT) W... Z... Y... X (EFFECT OUT) Keeping the commitment.
MUSIC: PULSING BEAT OUT...SEGUE TO THREE STINGERS
ANNOUNCER 1: News radio 1060.
The action takes place on a deserted city street, just after a brutal, late-night assault and robbery. The battered victim lies unconscious on the pavement. The assailant, Joe Badd, is running to his car and making his getaway.
The scene calls for the following sounds: ambience of a city street at night, footsteps, a car door opening and closing, the engine starting, and the car speeding off. These are common sounds easily recorded and even more easily obtained from prerecorded sound libraries (see Chapter 15). A typical, unimaginative sonic approach would be to produce the scene on a what-you-see-is-what-you-hear basis.
But without changing the action or the designated sounds, the audio can do more than just be there. The ambience, instead of simply being present, could be processed to sound dense and oppressive. The footsteps, instead of being those of ordinary shoes hitting the pavement, could be sonically fitted with sharp, lacerating, almost metallic attacks at each footfall. The car door handle could be snapped, the car door wrenched open, then sharply slammed shut with a report sonically similar to a high-pitched gunshot. The ignition could start with a grind, followed by a guttural revving and rasping acceleration of the engine. The drive-off could be punctuated by a piercing, brazen screech of tires. After all, these are not just any sounds; they are the sounds of nasty Joe Badd.
The school of documentarists producing in the cinéma verité style record life without imposing on it; production values do not motivate or influence content. Whatever is recorded has to speak for itself. Music, produced sound effects, and narration are renounced. Deciding what is recorded and edited are the only “manipulative” elements in the production process. Even within these guidelines, however, an imaginative sound designer still has creative leeway.
A documentary producer is doing a program about poverty in America. There are shots of various locales in which the poor people live, interviews with various families in their living quarters, and interviews with selected officials responsible for carrying out programs that theoretically help the underprivileged.
Urban and rural neighborhoods where the poor exist are usually depressed, dilapidated, and depleted. One approach to recording the ambient sound reflecting such conditions might be to produce an empty, forsaken, desolate quality. Conveying such an ambient quality isn’t a matter of simply sticking a microphone in the air and turning on the recorder. The imaginative sound designer will select a mic that produces a thin, transparent sound. Placement will be far enough away from reflectant surfaces so that the sound is diffused rather than concentrated and stable. On the other hand, a dense ambience created by placing a microphone with midrange coloration near reflectant surfaces might produce a more saturated, suffocating effect.
Interviews inside the living quarters could be miked to include room reflections, creating an overall hollowness to the sound. The microphone could be one that is flat through the upper bass and low midrange so as not to enhance the frequencies that produce a warm sound. Or the mic could be one with a peaky midrange response to create a tinny, spare sound.
Interviews with the officials who are dragging their bureaucratic feet could be miked to create a warm, close, secure sound, thereby contrasting the officials’ comfortable lifestyle with that of the underclass. Or mic selection and placement could be designed to a harsh, cutting sound, suggesting an insensitivity to poverty. Miking at a distance might suggest an official’s remoteness from the problem. All these approaches to the sound design enhance overall impact and meaning without compromising the style of cinéma vérité.
Sound in animated material is especially critical because it defines not only the action but the characters as well. This is particularly true in most TV cartoons produced today in which animated facial expressions and body movements are minimal. “Many experienced animators credit sound with contributing as much as 70 percent of the success of a project.”[8]
Chasing, stopping, falling, crashing, bumping, digging, and so on—all are defined by sound effects, music, or both. In animation, sound is created in ways that often exaggerate and accelerate the action; practitioners call it tune time. “You try to create ambiences, atmospheres, and sounds so that if you turned the picture off, you would probably think you were listening to a live action film.”[9] Although animation is a world unto itself, the audio design team has to keep in mind that the audience has expectations about what things should sound like.
That said, because sounds in animation are frequently exaggerated and accelerated, sound, more than appearance, gives most animated characters their delineation and personality. Think of Donald Duck, Daffy Duck, Scooby-Doo, Bugs Bunny, Roger Rabbit, Ariel (The Little Mermaid), Simba (The Lion King), and WALL-E to name a few.
Mr. Burns, in The Simspons, is a corporate antagonist—power hungry, evil, deceitful, and scheming. His sound, appropriately, is somewhat low in pitch and at times raspy and sibilant. Huckleberry Hound has a slow, evenly paced, low-pitched southern drawl to suggest the sense of control that is central to his characterization.
There can be sonic counterpoint as well. Tweety Bird is also in control, but his high-gpitched voice and small size suggest victimization. Wile E. Coyote’s sound suggests sophistication, wit, and culture, yet his actions indicate just the opposite.
Defining the sonic personality of a nonhuman character is usually not the purview of the sound designer, but the “ear” a designer brings to such a determination is invaluable to the director with sense enough to seek such assistance.
The audience viewing and listening to content on pocket- and handheld portable media players, such as cell phones and iPods, to say nothing of mediocre desktop and laptop computer sound systems, is burgeoning. In fact, a generation of users sees and hears media as much, if not more, on these small, mobile players than on conventional TV, film, radio, and music systems.
The quality of produced and reproduced sound today in film, radio, and high-definition television, and in recorded materials on DVD, Blu-ray, SACD, and high-density optical discs, is extraordinary and readily accessible to the consumer. The paradox is that with the potential to hear great sound through great systems, with too many listeners the opposite is happening: they are hearing their digital audio in compressed form, much of which reduces sound quality. And to exacerbate the situation, they are listening through inferior headphones or earbuds, often in noisy ambient environments, or through a less-than-sonorous computer speaker or computer loudspeaker system. If your goal is to become a professional in audio production, it is difficult to develop analytical and critical listening skills with such deficient exposure to sound.
This has also influenced the way audio is handled in production, depending on the intended distribution medium. Instead of being able to take advantage of audio’s potential for excellence, recordists and mixers are having to use broader sonic canvases: less detail and subtlety and reduced frequency response and dynamic range because the small, portable reproducing systems cannot handle the sound otherwise (see Chapters 18 and 23).
That said, and like it or not, the reality is that more and more audio (and video) is being produced with the mobile-media audience in mind. Therefore, designing sound for small, portable receivers involves considerations that are somewhat different from designing sound for traditional receivers. There are also limitations in how materials are produced for mobile media. (Production for mobile media is covered in Chapter 18.)
By their very nature, mobile media—portable devices capable of storing and playing digital audio, video, and images, such as cell phones, iPods, cameras, and laptop computers—can reproduce only so much; so to avoid pitfalls in realizing a sound design, it is necessary to consider what receivers cannot handle.
Detail is limited. Whereas in, say, conventional film, sound can consist of layers of information provided through vocal intonations, enhanced sound effects, and subtle, nuanced underscoring, or some combination of these components, the size (or lack thereof) of the speaker or the limitations of the earphones and earbuds cannot reproduce that type of detail to any worthwhile degree.
Because detail is reduced, so is perspective. There are constraints on creating proportional or subtle near-to-far and side-to-side aural imaging. There is only so much that can be done with a limited sonic palette. Picture also has the same problem with perspective due to the very small screen size.
Frequency response is limited. Many systems in mobile media essentially reproduce midrange because they cannot handle bass and treble frequencies. Midrange-only sound can be harsh and annoying. Even though the intelligibility of most sound resides in the midrange, there are many sounds in which the bass or treble are part of their recognizability and impact. In music, bass and treble are essential for an aesthetically pleasing experience.
Dynamic range is limited. Most mobile-media systems cannot handle anywhere close to a wide dynamic range. To reproduce sound with any effectiveness, it must first be compressed. Even the best compression schemes adversely affect some sound quality. A reduced dynamic range limits the strength, effectiveness, and impact of many types of sound effects. It certainly reduces the overall dynamics of music, particularly in larger ensembles.
With all that said, the sound designer also has to consider mobile-media systems that produce comparatively good sound through headphones and earbuds. This creates the challenge of the sound’s being too good and too high-energy for the tiny video image. The disproportional sound tends to separate from the video because the audio is oversized compared with the video. This is comparable to the aesthetic disconnect due to the sound from a small, monaural speaker in a TV receiver but in reverse—the video being oversized compared with the audio.[10]
The concepts developed in a sound design are realized in the actual production and post-production of the audio materials: sound effects, music underscoring, editing, premixing, and rerecording. These subjects are discussed throughout the rest of the book.
Sound design is the process of creating the overall sonic character of a production and is ongoing throughout the production process.
The sound designer is responsible for creative control of the audio—for putting a coherent sonic stamp on a production—although all members of the audio team make creative contributions to the soundscape.
In the parlance of audio, having “ears” means having healthy hearing and the ability to listen perceptively.
Having educated ears means the ability to listen with careful discrimination to style, interpretation, nuance, and technical quality in evaluating the content, function, characteristics, and fidelity of sound.
Learning how to listen begins with paying attention to sound wherever and whenever it occurs.
Analytical listening is the evaluation of the content and the function of sound.
Critical listening is the evaluation of the characteristics of the sound itself.
There are three domains to work with in creating a sound design: speech, sound effects, and music. Paradoxically, two possible additional domains include silence and the ability of sound to evoke a picture in the mind’s eye.
All sound is made up of the same basic components: pitch, loudness, timbre, tempo, rhythm, attack, duration, and decay.
Sound also has a visual component in that it can create pictures in the “theater of the mind.”
Sound has several functions in relation to picture: sound can parallel picture, sound can define picture, picture can define sound, sound and picture can define effect, and sound can counterpoint picture.
There is no set procedure for designing sound. At the outset the most important thing to do is study the script and analyze the auditory requirements line by line to determine the overall sonic approach to various scenes, for an entire work, or both.
Determining a sound design involves consideration of how the audience is to think or feel about a particular story, scene, character, action, or environment; from what point of view; and whether that is to be carried out mainly in the sound effects, the music, or both.
Determining a sound design also requires the awareness that doing so is often tantamount to defining a production’s conceptual and emotional intent.
The paradox in listening to recordings today is that with the potential to hear high-quality sound through great systems, with too many listeners the opposite is happening: they are listening to compressed audio through mediocre headphones or earbuds on mobile media or through a less-than-sonorous computer speaker or computer loudspeaker system.
[1] Walter Murch, “The Chase,” Music Behind the Scenes, Bravo, August 18, 2002.
[2] William Moylan, Understanding and Crafting the Mix: The Art of Recording, 2nd ed. (Boston: FocalPress, 2007), p. 90.
[3] Jeanette Thomas and Valerian Kuechle, “Quantitative Analysis of Weddell Seal (Leptonychotes weddelli) Underwater Vocalizations at McMurdo Sound, Antarctica,” Journal of the Acoustical Society of America, vol. 72, no. 6 (1982), pp. 1730-38.
[4] Michel Chion, Audio-Vision: Sound on Screen, trans. and ed. Claudia Gorbman (New York: Columbia University Press, 1994), p. 33.
[5] Randy Thom, “Designing a Movie for Sound,” Filmsound.org, 1999; www.filmsound.org/articles/designing_for_sound.htm.
[6] Kevin Hilton, “Walter Murch: The Sound Film Man,” Studio Sound, May 1998, p. 77.
[7] From Michael Ondaatje, The Conversations: Walter Murch and the Art of Editing Film (New York: Alfred A. Knopf, 2002), p. 118.
[8] Robin Beauchamp, Sound for Animation (Boston: Focal Press, 2005), p. 17.
[9] Bryant Frazer, “What Randy Thom Hears,” Film and Video, April 2005, p. 53.
[10] For more information about audio/video balance, see Herbert Zettl, Sight Sound Motion: Applied Media Aesthetics, 5th ed. (Belmont, Calif.: Thomson Wadsworth, 2008), pp. 334-35.