11

Sound Design

Films and games are both thought of as a visual medium, but the reality is that sound plays almost as important a part in the final product. In addition to tackling every piece of music within a game, a sound designer/engineer must also track down (and usually must record) every sound effect, piece of dialogue, and ambient sound within the game (this includes wind, water, city street, and so on). These are pretty much the same assets for which a sound designer would be responsible when making a movie. Because of this, many sound designers have worked in both areas, games and movies, as well as applying their trade to theatre productions and television.

Excluding the music, the sounds that you will import into the game will either be a sound effect (like an explosion or a spaceship taking off), a practical sound (car door shutting or footsteps), or ambient/room tone (crowd in a mall, applause, or wind blowing through the trees). Each of these types of sound has its own set of challenges in obtaining them.

Becoming an effective sound designer means having a great ear for hearing what the best possible sounds are for a specific project—as well as knowing how to find and obtain those sounds. There are many sources for these sounds—all of which will be discussed later in this chapter—but the key for success as a sound designer lies in matching an audio style with the visual theme of the project; this can be as simple as importing authentic sounds and music for a game set in the 1930s, or as difficult as creating all the sounds for a possible future within a sci-fi game. Sound design includes every sound within the game down to the basics—the most basic of all sounds within a game and film comes from the soundtrack and background music.

11.1Cinematic Music

Most of the time, sound designers work with the creative heads of a production to come to an agreement as to what type/genre of music fits the project best. In some cases, this decision has already been made by the game’s creative director and this choice has led to a particular sound designer. Though some designers work in all genres of music and styles, most specialize in a particular area—usually because most sound designers are musicians and play in a certain style. Some designers also segue into the field of sound design because of experience working as a sound engineer in a studio or have worked as a sound editor for a film. Either way, if the designer has the ability to compose and perform a certain type of music, this is an easy way to lower your budget in that department a bit—it means not contracting out to a composer or performer for your game’s music (or at least a portion of it).

If you have chosen a designer that does not compose music, or does not work in the style of music you want for the game, the sound designer will shop around for a composer or performer who is already writing in the style desired and is close to the expectations of the creative director. Music is one of the easiest ways to create a uniform tone within a game, so it is important that decisions made in regard to musical choices be done very carefully. Some of the things to consider when working with music are:

• What genre or style of music do you want?

• What is the music trying to do?

• How many different pieces of music do you need?

When working on a budget, hiring a sound designer who is skilled in creating electronic (or loop-based) music is a great choice. Most of these programs can be bought for very little cash and learning how to manipulate and create loops can be learned quickly. Also, most modern sound designers are accustomed to working on a computer for mixing and editing, so it is no great leap to incorporate loops.

A great and inexpensive alternative to composing electronic music is to use unsigned bands for the game’s music (if that type of music is going to fit the project). One trip to MySpace will educate you on the sheer amount of musicians working throughout the world. Pick ten or twenty bands that are playing the style of music you want to use and ask for permission to use one of their songs in your game—sometimes, you can even get an instrumental version of the songs from them, as most unsigned musicians retain the tracks from their recording sessions. Make sure you get a legal permissions form signed from them, though, before using the music in the game. Getting licensing from signed musicians/ composers takes quite a bit of time, so if you’re going to go this route, you may want to start this process as early as possible— probably in preproduction.

Once you have obtained the music for the game—with the guidance of the creative director—the sound designer can move on to working on the game’s other audio needs. Before you can actually record, though, you probably need a few pieces of equipment and software.

11.2Tools for Great Sound

Getting all the various audio assets for the game requires some specific equipment and software. Typically, most dialogue is recorded in an actual studio with the sound designer present—or at least a competent engineer who supervises the session and submits the recordings for approval. Sound effects and great ambient/room/environmental sounds must be obtained in the field. To accomplish this, you need a field recorder (probably one that records to digital audio tape) or sampler with a studio-quality microphone and headphones.

The equipment for recording can be quite expensive—chances are good that you will do your voiceover work in an external sound studio—so finding a sound designer who already owns his or her own gear for field recording is another way to trim the budget. Make this a consideration during hiring. The software, on the other hand, will have to be licensed by your production company, so you must make some careful decisions regarding these choices.

Most of the required software is computer-based recording programs, and there are quite a few of these on the market, so try them out before buying. You should consult with the sound designer as well to see if he or she has a preference or any experience working with a particular program. Some of the more prevalent programs include Pro Tools, Cakewalk Sonar, and Sony Soundforge. You also need an interface to connect the recorder to your computer to upload the sound files. Again, this is a common piece of equipment, so the sound designer may already own one.

The final consideration (though probably the first tackled) is how all the sounds will be imported into the game. For this, consult with the technical director or lead to determine the best audio format, compression, and so on for getting everything into the game. This may require the purchase of another piece of software (usually called “audio middleware”) and may be a factor when choosing the game’s engine. All of these choices should be made in preproduction along with the creative decisions regarding audio.

11.3Sound Effects and Sampling

There are several different ways to get all the audio assets you need for your game that have been used in the film industry for years—these include contracting out to independent audio/ sound personnel, buying prepackaged sound effects, and recording your own sounds in the field. Contracting out should be considered only if the sounds needed are unavailable otherwise, as that is the most costly method of obtaining what you need.

Prepackaged sound effects are available from many companies. You can usually download the files directly from the companies’ sites (just be sure to sample them first and make sure that they are of a high enough quality to use in the game) or purchase CDs of common sounds. These CDs are sold with names like “Sounds of the Jungle” or “Military and Weapon Sounds”. These types of sound effects can save you a lot of leg work when trying to record everything you need in the field. Once you have obtained everything you can using this cost-effective method, you can then tackle getting all the miscellaneous stuff you’ll have to record on site.

Going out on location to record means taking a small sound team, a recorder or sampler, and a microphone/headset to the actual locales where the sounds can be obtained. You may have to do this because the particular sound you want is nowhere to be found otherwise, or it may be a very specialized sound (say, rocks bouncing off a piece of plastic then hitting water) that simply has not been archived anywhere. Either way, going out to get the sounds is an inevitable task during development.

Once you get to the site, set up for recording the sound by making sure that you can duplicate the sound you need multiple times—though this may seem obvious, some sounds are difficult to obtain and getting one take is usually not enough. When this is done, you can then attach the headset and microphone to the recorder. One person should point the microphone as close as possible to the sound (make sure that this is a unidirectional microphone, like a shotgun mic, so that you will pick up the least amount of external noise as possible—unless of course you are shooting ambient or environmental sound, which would require a good omnidirectional microphone) while the other person monitors the recorder through the headset to make sure that the signal is strong and clear with no interference or outside noise.

Once you have obtained the sound you want, you can then remove the media (usually a digital audio tape or digital media card) and label it for download later at the studio. This is pretty much the process you will use to get everything on location. Before heading out, though, here are some questions you should consider:

1. How many different environments/locations are sounds needed from?

2. How many individual sounds will you need from each location?

3. How much space (media/tapes) do you need to capture several takes of each sound?

The logistics involved in good sound design is yet another aspect that should be discussed while making the game’s schedule and budget. Having the sound engineer or designer present when doing the script breakdown is another way to tally up the sounds and sound locations during preproduction.

11.4Effective Sound Design

Remember that as you develop and obtain all the audio assets you need for your game, the sound design for a game is an ongoing process. As the art department and engineers develop assets for the game, they turn to the sound designer to bring them to life. Thus the total number of sound effects and sounds for the game will be constantly increasing. It is important then to make sure that the sound designer understands how the other departments work and how they document assets that are being produced. Major locations will be identified early in the development cycle (hopefully during preproduction, in a script), so getting ambient sounds and general effects from those places can be done early, but the other thousand individual sounds that inevitably pop up must be gotten during the production cycle.

Treat the sound department just as you would any other in the production team and allow them to contribute to the wiki pages, do daily delta reports, and otherwise participate. Keeping the sound department integrated with development makes your production team run much smoother, keeps the sound design on point, and gets you the most effective and cinematic game possible.

Interview: Marc Schaefgen, Sound Designer

Newman: Sound is such a huge part of the cinematic experience. How do you approach working on a new project?

Schaefgen: With games, we have a unique situation in that we do not really get a postproduction phase. You are constantly chasing the tail of change during what would normally be a postproduction period. Some games have done it successfully, but it takes mad production skills and a team that can consistently hit deadlines, which is a rare combination. So, with that knowledge, you have to create a process that fits with the game style and the team dynamic. Each game process might be slightly different, but I generally approach them all from the same high-level mindset. Developers have many ways to slice the development pie, but most agree that game development has three very distinct phases: concept, prototype, and production. Audio should really be no different than any other part of the production and align their goals with the rest of the team. For the concept phase, those main goals should be to outline the game play and its systems, and to set the tone and vision. During the prototype phase, you are proving out those game play concepts while also demonstrating your ability to deliver on the overall vision or experience.

During the concept phase for audio, I like to create three main documents: the music, SFX [sound effects], and voice style guides. From those three main documents, subdocuments are created that help to further define those styles; each style requires a different approach to its documentation. For instance, from the music style guide, I create a music reference compendium. This compendium is composed of music samples taken from all types of media (TV games, film), that to me expresses the emotion, style, and instrumentation that is indicative of what I want the final score to be like. The important partner to this compendium is the reference guide that goes into detail about each music sample, why it was chosen, likes and dislikes, instrumentation notes, and any other pertinent information that I feel is appropriate. Do not be shy at this point. Spelling out your vision clearly to everyone involved is critical to moving forward and finalizing schedules and budgets. That is just the music angle to the project. For sound design and voice, I go down similar but different paths, defining the style. When it comes to voice, the writing plays a major factor in the vocal style, thus demonstrating the need for collaboration with the team leaders during every phase of development.

The prototype phase is akin to what film calls preproduction. In film, for the most part, your technology is standardized. You use the same cameras, software programs, compositing tools, and so on as the next guy. In games, not so much. Your technology is your mojo. Even developers that license their development technologies enhance and combine those technologies in ways that no other developer would. So this is the time to prove to the people that write the checks that your team has what it takes to make this product. You need to instill that confidence at all levels, and audio is no exception. By the end of the prototype phase (which can last more than a year, and sometimes multiple years) you should have what is commonly referred to as a “vertical slice”. That is, a small excerpt or demo of your product that demonstrates every major facet of that product and proves that you can build the rest of the product to a reasonable schedule and budget.

Production phase is where all the real hard work comes. This is where audio is churning right alongside everyone else. There are no shoots, or dailies, or dubs needed for rough cuts. We build the audio for the game as the game itself is being built. This requires solid scheduling of resources and at times bulking up staff temporarily either in-house or off-site, to ensure that those schedules can be met.

Newman: When balancing music with dialogue, what types of considerations do you take to make sure you get the best possible mix?

Schaefgen: As games are nonlinear, you sometimes have no idea when a line of dialogue is going to play, and what other sound sources at that time are already playing. When I start a project, I determine one of two ways to approach this issue. The first is the easiest, which is to use a ducking mechanism, so that when any line or dialogue or a selected line of dialogue is played, all other sounds (or a selected set of sounds) are reduced in volume by a certain amount. This trick ensures that those lines of dialogue will be audible in the mix no matter what other sounds are present. Because this is done systemically, it is less time-consuming for the mixer to balance properly.

The second method, which is more difficult, but also my preferred method, is to create a mix where the dialogue is always going to be audible. This might entail periodic uses of ducking, but not in a systemic fashion. It requires an extensive amount of testing and tweaking, consistently playing the game, tweaking levels, playing the game, tweaking levels, until the production team makes you stop. This method isn’t as efficient or fun, but the results are more compelling in my opinion. When you use ducking systemically, it has a tendency to be distracting and pull the user out of the immersion of the experience. I have used ducking to success in a situation where all dialogue was delivered by changing the screen to a letter-box format to evoke a cinematic vibe to the delivery. This change in visual made the change in audio acceptable.

Newman: Speaking of music, how do you determine the most appropriate style or genre of music to use to fit the mood of a project?

Schaefgen: This comes about during the concept phase and is largely a team effort. Game development is about working with a team. During the concept phase, the team is smaller and generally composed of the major decision makers of the project, which makes collaboration much easier. I am constantly talking to the game designers and artists about what their visions are and reading documents that they create to backup their visions, just like I am doing with audio. I dissect, examine, reread, ask questions, call meetings—everything I can do to gain the most information about what everyone is converging on in terms of vision and style.

Armed with that knowledge, I look to my personal taste and how I would want the score to evolve. Then I also look at the competition; what my peers in this genre are doing stylistically. If I see developers taking chances, I might be inclined to be more risky with my score style. If I see them being conservative and complacent, do I follow suit, or really break the mold? It all depends on the combination of project style, genre style, trends, and what I think will help immerse the player in the game experience. It all comes down to that one thing though—what is best to enhance the game play experience—all else pales in importance.

From there, I dig up a small quantity of music samples that I think hit the overall style and tone. Then I play those for the music stakeholders on the team and we go from there. This can be a long process of listening and digging for tunes, and listening some more. Getting those stakeholders to sit in a room together and be silent to listen to their musical choices is very important to them signing off on the style. Once we all agree, I build the rest of the music reference compendium and send that out to the stakeholders for feedback. One more time I make them sit in a room, listen, and comment. This listening time not only engages the stakeholders, but gives me the ability to understand their wants and needs and how to best accommodate the project.

Newman: When working with any form of live music (orchestra, band, and so on), what are some of the factors that influence the quality of the score?

Schaefgen: Emotion, emotion, emotion. A machine cannot emote the way a human does. Also, not every musician is going to emote in exactly the same way. Having a singular violinist play many parts all multitracked together in a small studio room will never sound like a string section in a beautiful-sounding hall where each musician contributes their own style and emotion to the music. But then again, it all depends on your musical style. For BlackSite: Area 51, we chose to not use live players in order to put that budget toward a lengthier score. It worked well for that game, because stylistically electronic instruments were a big part of the sound palette. Were it a purely orchestral score, then having live players would have greatly enhanced the emotional delivery.

In games, it is a bit of a trade-off and is constantly looked at as an area where the budget can be trimmed. Luckily, the level of quality demanded by the consumer is constantly growing. In film, no big-budget title would consider not using an orchestra to record the organic parts of the score—it is too common and not even a consideration. In games, however, it is not the norm and therefore scrappy game designers and audio directors fight the battle for quality at every revision of the budget spreadsheet. Once the consumer demand is high enough, developers of big-budget games will have to use live orchestras to remain competitive.

Newman: A lot of the challenges involved with good sound design seem to originate with special effects. How do you tackle recording sound effects on a location?

Schaefgen: Gathering decent source material is always a challenge for sound designers, but it also one of the more fun parts of the job. Midway has a commercial sound library that totals more than 800 GB; our proprietary internal library is much smaller, less than 100 GB. We are constantly striving to find new unique sound sources for our products. We have field recording gear, portable recorders, microphones, and accessories that we use to gather unique sounds out in the world. We have had a lot of construction around our offices in the last two years, so we have taken advantage of that in our field recordings. We have done gunshot recordings, car recordings, recordings of sounds in the desert, the city streets—anywhere we can find appropriate material. My team has open ears and is always on the listen for new and unique sounds.

Newman: Listening to the audio track of the game Blacksite: Area 51, it becomes obvious that there are specific musical cues that set the tone for the action that follows. Explain the process of determining where these cues should occur and how to implement them.

Schaefgen: For BlackSite, we decided to use the character classes as a basis for creating the musical themes that would weave in and out and create the main score. Also, each character class had unique instrumentation that created the sonic signature in the score. There were three main classes: humans (more specifically, soldiers), aliens, and alien/human hybrids. Depending on the dominant races for a given game-play scenario, we used a mix of themes and instrumentation that would cover those classes. Because the game was fairly linear in nature, we generally knew what was coming next, based on geographic location, so we used triggers placed in the world to cue music. Music was also controlled based on game states or AI [artificial intelligence] states. We didn’t want wall-to-wall music, so between encounters we opted for either no music or very low ambient cues.

Newman: The game and film industries both rely on the quality of audio immensely. What lessons have you have learned from the film industry that would help a new sound designer?

Schaefgen: A lot of game audio production is the same as film. We use Pro Tools and scads of plug-ins, we record our own source material, we work closely with the visionaries of the project to enhance that vision through sound, and so on. It didn’t use to be that way. Before digital audio was the norm and processing power was plentiful, developers synthesized all of the sound content for a game. So our process of making games hasn’t been influenced a whole lot by film, but the process for making the content that goes into the game has. Like I mentioned earlier, we don’t really get a postproduction phase. The game development cycle is very different from that of a film production, but it is still media. Media that is digitized and manipulated to evoke some sort of response from the player.

In this day and age, I feel that film has a lot more to learn from games than the other way around. The game industry had been studying the film industry for so long that they thought they wanted to be just like film, but soon after, realized the differences are there for a reason and that games are a unique artform and a totally viable means of expression on their own.

Newman: What advice do you have regarding getting the best possible finished project?

Schaefgen: Assemble the strongest team possible and let those people do what they do best. Facilitation is more important than ego. Get out of the way and provide the support they need to be successful. Communication is the next most important factor. Once your team is empowered, they need to be efficient and informed. Establishing and maintaining strong lines of communication between all disciplines is the key to success. Everything after that is gravy.