Sound influences how we create meaning. While language and the spoken word are in some respects most obvious, nonverbal speech, sound effects, and music also inform our capacity to interpret and understand dramatic intention. The subtleties and nuances of the human voice can communicate more about characters and their motivation than what the words alone might signify. Well-crafted and appropriate sound effects lend realism and convincing illusion to a production while music adds emotional depth and dimension to our experience as an audience.
All sound—speech, sound effects, and music—is made up of the same basic elements: pitch, loudness, timbre, tempo, and rhythm. Furthermore, every sound has its own “life cycle” or sound envelope comprising an attack, duration, and decay. During audio production, everyone involved is dealing with these elements, consciously or subconsciously, and assessing their effects on perception. Because each element contains certain characteristics that affect our response to sound, understanding those effects is fundamental to the art and craft of sound design. The following outlines some of the associations and responses that the different parameters of sound may elicit.
Pitch refers to the highness or lowness of a sound—its frequency. High-pitched sound often suggests something delicate, bright, or elevated; low-pitched sound may indicate something sinister, strong, or peaceful.
Loudness, or dynamic range, describes the relative volume of sound—how loud or soft it is. Loud sound can suggest closeness, strength, or importance; soft sound may convey distance, weakness, or tranquility.
Timbre is the characteristic tonal quality of a sound. It not only identifies a sound source—reedy or brassy—but also sonic qualities such as rich, thin, edgy, or metallic. Reedy tonal qualities produced by a clarinet or an oboe, for example, can suggest something wistful, lonely, or sweet. A brassy sound can imply something cold, harsh, fierce, bitter, forceful, martial, or big.
Tempo refers to the speed or pace of a sound. Fast tempos agitate, excite, or accelerate; slow tempos may suggest monotony, dignity, or control.
Rhythm relates to a sonic time pattern. It may be simple, constant, complex, or changing. A simple rhythm can convey deliberateness, regularity, or lack of complication. A constant rhythm can imply dullness, depression, or uniformity. Rhythmic complexity suggests intricacy or elaborateness. Changing rhythms can create a sense of uncertainty, vigor, or the erratic.
Attack—the way a sound begins—can be hard, soft, crisp, or gradual. Hard or crisp attacks can suggest sharpness, excitement, or danger. Soft or gradual attacks can imply something gentle, muted, or blasé.
Duration refers to how long a sound lasts. Sound short in duration can convey restlessness, nervousness, or excitation; more sustained sounds can create a sense of peace, persistence, or fatigue.
Decay—how fast a sound fades out—can be quick, gradual, or slow. Quick decays can create a sense of confinement, closeness, or definiteness; slow decays can convey distance, smoothness, or uncertainty.
Of course, these elements are not heard individually but in combination. Other aspects of sound, such as changing pitch, changing loudness, and acoustic interactions, also affect response. That these characteristics are elemental in sonic structure is not to suggest that sound design is prescriptive or developed by applying formulas; the basic components of sound structure do not occur separately but together in myriad forms. Rather, it is to introduce and define the building blocks of sound from which the sound designer shapes aural structure and meaning.
For an actor, effectively delivering the words from a page—whether they are in a commercial, documentary, or drama—involves more than just reciting them. It is often not what is said but how it is said that shapes the meaning. Examples of how nonverbal speech influences meaning include emphasis, inflection, speech patterns, pace, mood, and accent.
Emphasis—stressing a syllable or a word—is important to all speech, whether voiceover, narration, or dialogue. It often conveys the gist of what is said. Consider the question, “What were you thinking?” With no particular emphasis, the question is straightforward and might mean, “What is on your mind?” If, however, any word in the phrase is emphasized, a simple question may become an admonition or an accusation. The words remain the same; the aural emphasis alters the message.
Inflection—altering the pitch or tone of the voice—can also influence verbal meaning. By raising the pitch of the voice at the end of a sentence, a declarative statement becomes a question. Put stress on it, and it becomes an exclamation.
Speech patterns, cadence, or rhythm are important to natural-sounding speech and creating a believable character. A writer must be equally aware of what words mean and how they should sound. If the speaker is formal, vocabulary and sentence structure should be precise, and speech rhythms should sound even and businesslike. Informality would sound looser, more relaxed, and more personal.
The pace or tempo of spoken words can communicate nonverbal information about a character and the emotional tenor of a given situation. Halting, hesitant dialogue conveys a very different sensibility than lines delivered smoothly and slowly or those that are breathless and quick.
Sound can be evocative and affects the mood or feeling of words and sentences. Consider the following line from Keats’ Ode to a Nightingale, “The murmurous haunt of flies on summer eves.”[1] The sounds of the words themselves evoke the din of insects on a summer night.
Sound effects (SFX) can be classified as anything sonic that is not speech or music. They are essential to storytelling, bringing realism and added dimension to a production. It is worth noting that the art of sound effects design is about creating not only something big or bizarre but also crafting subtle, low-key moments. Good effects are about conveying a compelling illusion. In general, sound effects can be contextual or narrative. These functions are not mutually exclusive; they allow for a range of possibilities in the sonic continuum between what we listen for in dialogue—namely, sound and words to convey meaning—and what we experience more viscerally and emotionally in music.
Contextual sound emanates from and duplicates a sound source as it is. It is often illustrative and also referred to as diegetic sound. The term derives from the Greek, diegesis, meaning to tell or recount and, in the context of film, has come to describe what comes from within the story space. If a gun fires, a car drives by, or leaves rustle in the wind, what you see is what you hear; the sound is naturalistic in structure and perspective.
Narrative sound adds more to a scene than what is apparent and so performs an informational function. It can be descriptive or commentative.
As the term suggests, descriptive sound describes sonic aspects of a scene, usually those not directly connected with the main action—but key in adding to the compelling nature of the illusion. Descriptive sound for a village scene set in the Middle Ages might include oxcarts lumbering by, the sounds of a tinker repairing a pot, a distant church bell tolling, and the indistinguishable hubbub of human activity in a market square.
Commentative sound also describes, but it makes an additional statement, one that usually has something to do with the story line. For example: An aging veteran wanders through the uniform rows of white crosses in a cemetery. A wind comes up as he pauses before individual markers. The wind is infused with the faint sound of battle, snippets of conversations, laughter, and music as he remembers his fallen comrades.
Sound effects have specific functions in how they influence meaning within the general contextual and narrative categories. They break the screen plane, define space, focus attention, establish locale, create environment, emphasize and intensify action, depict identity, set pace, provide counterpoint, create humor, symbolize meaning, and create metaphor. Paradoxical as it may seem, silence is also a functional sound effect.
A film or video without sounds, natural or produced, detaches an audience from the onscreen action. The audience becomes an observer, looking at the scene from outside rather than made to feel a part of the action—at the center of an acoustic space. The presence of sound changes the audience relationship to what is happening on screen; it becomes part of the action in a sense. The audience becomes a participant in that there is no longer a separation between the listener-viewer and the screen plane. Therefore, it can also be said that sound effects add realism to the picture.
Sound defines space by establishing distance, direction of movement, position, openness, and dimension. Distance—how close or far a sound seems to be—is created mainly by relative loudness, or sound perspective. The louder a sound, the closer to the listener-viewer it is. Thunder at a low sound level tells you that a storm is some distance away; as the storm moves closer, the thunder grows louder.
In shots, other than close-ups, in which a number of elements are seen at the same time, how do you know where to focus attention? Of course directors compose shots to direct the eye, but the eye can wander. Sound, however, draws attention and provides the viewer with a focus. In a shot of a large room filled with people, the eye takes it all in, but if a person shouts or begins choking, the sound directs the eye to that individual.
Sounds can establish locale. A cawing seagull places you at the ocean; honking car horns and screeching brakes place you in city traffic; the squeak of athletic shoes, the rhythmic slap of a bouncing ball, and the roar of a crowd place you at a basketball game.
Establishing locale begins to create an environment, but more brush strokes and textures are often needed to complete the picture. Honky-tonk saloon music may establish the Old West, but sounds of a blacksmith hammering, horses whinnying, wagon wheels rolling, and six-guns firing create environment.
Sounds can emphasize or highlight action. A person falling down a flight of stairs tumbles all the harder if each bump is accented.
Whereas emphasizing action highlights or calls attention to something important, intensifying action increases or heightens dramatic impact. A car’s twisted metal settling in the aftermath of a collision emits an agonized groaning sound. In animation, sound (and music) intensifies the extent of a character’s running, falling, crashing, skidding, chomping, and chasing.
Depicting identity is perhaps one of the most obvious uses of sound. Barking identifies a dog, slurred speech identifies a drunk, and so on. But on a more informational level, sound can also give a character or an object its own distinctive sound signature: the rattle sound of a rattlesnake to identify a slippery villain with a venomous intent; thin, clear, hard sounds to convey a cold character devoid of compassion and so on.
Sounds, or the lack of them, help set pace. The incessant, even rhythm of a machine creates a steady pace to underscore monotony. The controlled professionalism of two detectives discussing crucial evidence becomes more vital if the activity around them includes such sonic elements as footsteps moving quickly, telephones ringing, papers shuffled, and a general hubbub of voices.
Sounds provide counterpoint when they are different from what is expected, thereby making an additional comment on the action. A wounded soldier lying on a bed and looking at a ceiling fan’s rotating blades that sound like those of a helicopter suggest the battlefield he has just left.
Sounds can be funny. Think of boings, boinks, and plops; the swirling swoops of a pen-nywhistle; the chuga-chugaburp-cough-chuga of a steam engine trying to get started. Comic sounds are indispensable in cartoons in highlighting the shenanigans of their characters.
Sound can be used symbolically. The sound of water swirling down a drain is heard as a murder victim slumps to the floor of a bathtub.
Sound can create a metaphorical relationship between what is heard and what is seen. A wife leaving her husband after enduring the last straw in a deeply troubled marriage walks out of their house, slamming the door. The sound of the slam has the palpable impact of finality and with it there is a slowly decaying reverberation that reflects a married life that has faded into nothingness.
Silence is not generally thought of as sound per se—that seems like a contradiction in terms. But it is the pauses or silences between words, sounds, and musical notes that help create rhythm, contrast, and power—elements important to sonic communication.
In situations where we anticipate sound, silence is a particularly powerful element. A horrifying sight compels a scream—but with the mouth wide open there is only silence, suggesting an unspeakable horror. The silence preceding sound is equally effective. A pistol shot breaks the quiet of winter night. Similarly, silence can be dramatic following sound. An explosive device is set to go off in a crowded city square. The ticking of the bomb gets louder as it reaches detonation time. Then, silence.
Music is elemental. Its sonic combinations are infinite, and its aesthetic values fulfill basic human needs. Therein lies its power in underscoring—adding background music that enhances the content of a scene by evoking a particular idea, emotion, point of view, or atmosphere. Music is generally thought of as an art form that evokes emotion. That is true, of course. It provides affective information—information related to feeling and mood. But it also provides cognitive information—information related to mental processes of knowledge, reasoning, memory, judgment, and perception.
Underscore music is original or library music. Unlike sound effects, underscoring is nondiegetic sound in that it comes from outside the story space. Underscoring serves picture in a number of ways. As composer Aaron Copland observed many years ago, and which still holds true today: Underscoring creates a more convincing atmosphere of time and place; it underlines psychological refinements—the unspoken thoughts of a character or situation; it serves to fill pauses between conversation; it builds a sense of continuity; it underpins dramatic buildup and gives finality to a scene.[2]
Music has the same basic structural elements common to all sound, such as pitch, loud-ness, tempo, tone color, and envelope. It also contains other characteristics that broaden its perceptual and aesthetic meaning such as melody and tonality; harmony and its qualities of consonance, dissonance, and texture; tempo; dynamic range that is quite wide compared with speech and sounds; and style in limitless variety. The characteristics of music are both more and less complex than those of verbal language. Unlike speech (and sound effects), musical structure is both horizontal and vertical and therefore at once linear and simultaneous. That is, linear sound provides melody and rhythm; simultaneous sound provides harmony and texture. The entire audible spectrum of sound can be used in an infinite variety of combinations.
Melody is a succession of pitched musical tones of varied durations. Because each tone has duration, melody also establishes rhythm. Hence, melody has both a tone quality and a time quality and cannot be separated from rhythm. Generally, if a melody moves in narrowly pitched steps and ranges, it tends to be expressive and emotional. If it moves in widely pitched steps and ranges, it tends to be conservative and unexpressive. Melodies are usually written in keys or tonalities, designated as major and minor. Subjectively, keys in the major mode usually sound positive, happy, bright, and vigorous; keys in the minor mode usually sound darker, tentative, wistful, and melancholy.
Harmony is a simultaneous sounding of two or more tones, although three or more tones are usually necessary to be classified as a chord. Chords are categorized as consonant or dissonant, and they are important to musical texture. Consonance in music is produced by agreeable, settled, balanced, stable-sounding chords. Dissonance is produced by unsettled, unstable, unresolved, tense-sounding chords. Texture, as the term suggests, is the result of materials interwoven to create a fabric. In music, melody and harmony are those interwoven materials. Undoubtedly, the infinite variety of musical textures that can be designed is a major reason for music’s universal appeal and role in human communication.
Tempo provides the pulse and the drive in music. A quick tempo tends to intensify stimulation; a slow tempo tends to allay it.
Music underscoring performs many of the same functions in audio design as sound effects but more broadly and diversely. One essential difference between sound effects and music is that sound effects are generally associated with action and music with reaction. This can be argued, of course, but it serves to provide insight into their different roles and effects. The unique language, vast vocabulary, and universal resonance of music also make it so powerful and so widely applicable in aural communication.
Many musical styles and themes are indigenous to particular regions. By recalling these styles and themes or by simulating a reasonable sonic facsimile, music can establish a locale, such as an Indian village or Rio de Janeiro during Carnival.
Music emphasizes action by defining or underscoring an event. For example, a dramatic chord underscores shock or a moment of decision or tempo increasing from slow to fast emphasizes impending danger.
Music intensifies action, usually with crescendo or repetition. The scariness of sinister music builds to a climax behind a scene of sheer terror and crashes in a final, frightening chord. The repetition of a short melody, phrase, or rhythm intensifies boredom, the threat of danger, or an imminent action.
Music can identify characters, events, and programs. Think of John Williams’s score for Jaws with its distinct musical phrase associated with the menacing shark. A particular theme played during an event identifies the event each time it is heard. Themes also have long served to identify radio and television programs, films, and personalities.
Music sets pace mainly through tempo and rhythm. Slow tempo suggests dignity, importance, or dullness; fast tempo suggests gaiety, agility, or triviality. Changing tempo from slow to fast accelerates pace and escalates action; changing from fast to slow decelerates pace and winds down or concludes action. Regular rhythm suggests stability, monotony, or simplicity; irregular (syncopated) rhythm suggests complexity, excitement, or instability. Using up-tempo music for a slow-moving scene accelerates the movements within the scene and vice versa.
Music that provides counterpoint adds an idea or a feeling that would not otherwise be obvious. Football players shown blocking, passing, and running are counterpointed with ballet music to underscore their grace and coordination.
A sliding trombone, clanking percussion, or a galumphing bassoon can define a comic highlight or underscore its humor.
Among the many uses for musical style is fixing time. Depending on the harmonic structure, the voicings in the playing ensemble, or both, it is possible to suggest ancient Greece, medieval France, the Roaring Twenties, 1960s San Francisco, the future, times of day, and so on.
If music can be used to fix a period in time, it can also be used to recall a past event or foretell a future occurrence. For example, a theme used to underscore a tragic crash is repeated at dramatically appropriate times to recall the incident.
Perhaps no other form of human communication is as effective as music in providing atmosphere, feeling, or mood. There is a musical analog for virtually every condition and emotion. Music can evoke atmospheres that are thick, unsavory, cold, sultry, and ethereal. It can evoke feelings that are obvious and easy to suggest, such as love, hate, and awe, as well as subtle feelings such as friendship, estrangement, pity, and kindness. Music can convey the most obvious and the subtlest of moods: ecstasy, depression, melancholy, and amiability.
The following example demonstrates how sound design can affect the meaning of the picture.
FADE IN
INTERIOR. BEDROOM. NIGHT. A LITTLE GIRL IS LYING ON A BED LOOKING AT THE CEILING. A SHAFT OF MOONLIGHT COMES IN THROUGH AN OPEN WINDOW ILLUMINATING ONLY PART OF HER FACE. THE CAMERA SLOWLY ZOOMS IN.
Clearly, the script’s story and the director’s choices would govern the sound design for this shot. But the intent of this example is to demonstrate sound’s significant influence on picture. Consider the shot with the following sound patterns and how each pattern influences its meaning and feel.
The sounds of conversation and revelry coming from another room
The sounds of a man and a woman arguing in an adjacent room and a sudden crash
Two men are heard whispering offscreen
An old woman’s voiceover describes childhood memories
The girl breathing, humming softly—almost imperceptibly—to herself
The thrum of an air conditioner and the polyrhythmic din of insects
Gentle breaking of waves and a distant foghorn
The whine of air raid sirens, the rumble of airplanes, and the dull thudding of distant bombs
A single dog howling, building to a scattered chorus of baying hounds in the distance
The creaking of floorboards and the sound of a door latch opening
The distant sounds of a playground with children laughing and singing
Street traffic, the occasional car horn, and the booming bass of music from a passing car
This example has other thematic possibilities, of course.
The choices of dialogue and narration open up numerous avenues for the audience to interpret the scene differently. In addition to what is said, aspects of emphasis, inflection, patterns of speech, pace, mood, and accent also suggest a wide range of dramatic possibilities.
An audience hearing the muffled argument between a man and a woman through the walls may not hear specific words, but the nonverbal dimension of their conversation communicates the tension between them. Their voices rise and lower in intensity, the tempo of the exchange varies, and punctuated silences maintain a level of suspense.
If we think of the scene and hear the voiceover of an old woman reminiscing about her youth haltingly and deliberately with a Russian accent and a twinge of melancholy, it tells us one story. Change the voice to an American from the rural South who speaks quickly with a pronounced drawl and laughs to herself as she pauses between sentences, the scene changes—even though both are talking about childhood memories.
The various ambient soundscapes and sound effects palettes contribute to our understanding of the scene and how we make sense of characters’ actions and intentions. The arguing couple heard against the rhythms of a hot summer night in the country carries a different sensibility than that of the inner city—what we hear tells us about time, place, and circumstance.
The Russian woman’s reminiscences juxtaposed against the sounds of distant battle as opposed to ocean surf and foghorn convey a different backdrop for the narration. In these examples, what we hear beyond the content of the dialogue in terms of ambience and sound effects informs our capacity to interpret and shape meaning.
Now consider what music underscoring does to the scene, with or without the sound effects and any number of variables with regard to dialogue and/or narration. With music that is, say, celestial, mysterious, sinister, playful, comedic, romantic, melancholy, blissful, animated, or threatening, notice how the idea and the feel of the scene changes yet again. In the same vein as these musical suggestions, consider the wide range of musical styles and how these might influence our interpretation of the scene.
For the man and woman engaged in a heated argument, music can heighten the conflict— whether we are hearing hip-hop, bebop, or a piece of atonal music from a string quartet. With the Russian woman, the plaintive sound of a solo balalaika might lend an intimacy to her voiceover, while the strains of an orchestra or choir may give us a sense of pathos. Each choice brings with it a shading of nuance and information that pulls the audience more deeply into the drama of the moment.
The effective interplay of dialogue and narration—quite apart from what the words might signify—along with sound effects and music all contribute to how we create and interpret meaning and dramatic intent.
All sound is made up of the same basic components: pitch, loudness, timbre, tempo, rhythm, attack, duration, and decay.
Influences of nonverbal speech on meaning include emphasis, inflection, speech pat-terns, pace, mood, and accent.
Sound effects (SFX) can be classified as anything sonic that is not speech or music. They help amplify the reality created in a production.
Sound effects perform two general functions: contextual and narrative. Contextual sound emanates from and duplicates a sound source as it is. It is also known as diegetic sound—coming from within the story space. Narrative sound adds more to a scene than what is apparent.
Narrative sound can be descriptive or commentative.
Sound effects can break the screen plane, define space, focus attention, establish locale, create environment, emphasize and intensify action, depict identity, set pace, provide counterpoint, create humor, symbolize meaning, and create metaphor.
Silence can be used to enhance sonic effect.
Music underscoring consists of adding background music that enhances the content of a scene by evoking a particular idea, emotion, point of view, or atmosphere.
Underscore music is original or library music added to enhance a scene’s content.
Music has the same basic structural elements common to all sound such as pitch, loudness, tempo, tone color, and envelope. It also contains other characteristics that broaden its perceptual and aesthetic meaning such as melody and tonality; harmony and its qualities of consonance, dissonance, and texture; tempo; dynamic range that is quite wide compared with speech and sounds; and style in limitless variety.
Melody is a succession of pitched musical tones of varied durations.
Harmony is a simultaneous sounding of two or more tones, although three or more tones are usually necessary to define a chord.
Tempo provides the pulse and the drive in music.
The dynamic range of music can be much wider than the dynamic ranges of speech and sounds.
Style is a fixed, identifiable musical quality uniquely expressed, executed, or performed.
Music underscoring provides such functions as establishing locale, emphasizing action, intensifying action, depicting identity, setting pace, providing counterpoint, creating humor, fixing time, recalling or foretelling events, and evoking atmosphere, feeling, and mood.
Sound design can significantly influence the meaning and feel of a picture.
[1] Sir Arthur Thomas Quiller-Couch, The Oxford Book of English Verse (Oxford: Clarendon, 1919, [c1901]); Bartleby.com, www.bartleby.com, 1999.
[2] Aaron Copland, Our New Music (New York: McGraw-Hill, 1941).