No game is complete without sound. This includes both background music and sound effects. Playing a sound effect in any OS is a pretty simple thing, but making sure your application or game plays nice with the rest of the iOS system can be a little tricky. If you consider that the game should allow users to play their own music, respond to interrupts and notifications, and honor the Silence switch, you can see that there is more to it than just playing an audio track.
In this chapter we will explore how to make sure your game is a good citizen on iOS, as well as the basics for playing sound. We will not be covering advanced sound concepts or OpenAL, as those are sophisticated topics and we don’t have the space to do them justice here. The project Sample 11 contains the code for this chapter.
Before we get into some of the details of a full application, let’s quickly look at the basic class for playing audio, AVAudioPlayer. Listing 11-1 shows the simplest code to play a sound.
Listing 11-1. AVAudioPlayer Example
NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@" viper_explosion" ofType:@"m4a"]];
AVAudioPlayer* player = [[AVAudioPlayer alloc] initWithContentsOfURL:url error: nil];
[player play];
In Listing 11-1, we see the AVAudioPlayer player, instantiated with an NSURL. The NSURL is constructed from the applications main NSBundle for the file viper_explosion.m4a. The file viper_explosion.m4a is simply a file added the Xcode project. In order to actually play the sound, we call play on player. This plays the audio once at the default volume and pan. There are a number of ways we can improve on this basic code to make it more suitable for a game. But before we get into those details, let’s understand how sound in an iOS application works in terms of being a good citizen on the iPhone or iPad.
Apple’s Human Interface Guidelines explain how they expect applications to behave in terms of audio. This is worth a read at some point, and can be found here: http://goo.gl/L0vYH.
Without going over Apple’s guidelines point by point, let’s look at some of the highlights in regard to how they apply to games. Before we get started, I must say that these points seem to apply more to the iPhone than to the iPad, since the usage of the iPhone is more sensitive. People always have their phones on them and often are in social situations that demand a specific behavior. If you are at a wedding or some other event, you really don’t want your phone to start beeping even when you have the Silent switch set to silence. Which brings us to our first point.
Users Switch Their Devices to Silent When They Want To
The switch on the iPhone or iPad that turns off the ringer is called the Silent switch. Although it is possible to write an iOS application that plays sound even though this switch is set to silent, I can’t think of any circumstances that would justify doing so. There may be exceptions, but you would need a really good reason. For most run-of-the-mill time-waster games (my favorite), I want to be able to kill sound. In fact, I play almost all of my games on silent.
There is one exception to this rule, however. If the user has headphones on, then the game is expected to play its audio regardless of the setting of the Silent switch. In addition, if users have their headphones on, they expect the audio for the game to come out of the headphones. Sounds obvious, but there are many games and applications that do this wrong.
The last point on this topic concerns the volume controls for the device. A game should honor the volume levels of the phone. So when users adjust the buttons on the side of the device to lower the volume of the game, the volume should go lower. Again, it sounds obvious, but paying attention to these little details will ensure your game plays well with the rest of the system and meets user expectations.
Your Game May Not Be the Only Application Making Sound
Many users like to play games while listening to music or other audio that’s playing through their iOS devices. It is important that your game allow this other audio to play while your game is running. Apple recommends that a game (or other application) in which the sound is not a critical part of the experience should remain silent if other audio is playing. Personally, I deviate slightly from this philosophy. If your game has background music, then I agree that the audio track should yield to the user’s music or podcast. However, there are other types of sound in a game that might make sense to play over the user’s audio. Perhaps a sound that warns the user of a boss fight might be a good exception to this rule. You can make the case that some sounds should be played regardless of which background music the user has selected.
Your Game Is Not the Only Application
In addition to honoring the background music, the game should consider the different ways that the game can be interrupted and still behave correctly. Consider the ways that a game may be interrupted:
- The user exits the game with the Home button.
- The user double-clicks the Home button to reveal the background applications.
- The user does a four-finger swipe (iPad only) to switch to another application.
- The user receives a popup warning, such as low power.
- The user receives a phone call.
All of these events can be summarized as the game’s resigning its active status. Conversely, when the user returns to the game, it is said to become active. These ideas, of course, are captured in the tasks applicationWillResignActive: and applicationDidBecomeActive: of the class UIApplicationDelegate. In the following section, we will review how we use these tasks to ensure correct application behavior.
Implementing Sound in Your Game
We have discussed some of the finer points of audio in an iOS game. We have looked at the expected behavior concerning the hardware controls, headphones, and different ways a game interacts with other applications. In this section, we will explore how these concerns are addressed in code. We will also look at how the class GameController can be extended to provide a simple audio framework for your game.
The first thing that any game should do regarding audio is to define how audio should be played by the application. iOS offers many different ways that applications can play sound to support different categories of applications. For example, the built-in Music application plays audio while other applications are active. Further, a sound-editing application or Garage Band has different requirements. A game application has yet a different set of requirements. Listing 11-2 shows how a game should initialize its audio session.
Listing 11-2. initializeAudio: (GameController.m)
-(void)initializeAudio{
AVAudioSession* session = [AVAudioSession sharedInstance];
[session setCategory: AVAudioSessionCategoryAmbient error: nil];
[session setDelegate:self];
UInt32 otherAudioIsPlayingVal;
UInt32 propertySize = sizeof (otherAudioIsPlayingVal);
AudioSessionGetProperty (kAudioSessionProperty_OtherAudioIsPlaying,
&propertySize,
&otherAudioIsPlayingVal
);
otherAudioIsPlaying = (BOOL)otherAudioIsPlayingVal;
if (otherAudioIsPlaying){
[backgroundPlayer pause];
} else {
[backgroundPlayer play];
}
}
In Listing 11-2, the first thing we do is get a reference to an AVAudioSession by calling sharedInstance. We assign a category to this application by calling setCategory and specifying AVAudioSessionCategoryAmbient. This indicates that the game is willing to allow audio from other applications to play (such as the user’s music) and that the audio from this application should be mixed with the audio from the other application.
Once we indicate the category of this application, we test to see if the audio is playing. We do this by calling the C function AudioSessionGetProperty and populating the UInt32 otherAudioIsPlayingVal with either a 0 or 1. We cast this UInt32 to a more Objective-C friendly BOOL and store the value in otherAudioIsPlaying.
Lastly, we either play or pause our background music depending on whether the other audio is playing. We do this by calling the appropriate task on backgroundPlayer, which is an AVAudioPlayer defined elsewhere. (We will get back to backgroundPlayer later in this chapter.)
initializeAudio is called in the doSetup task of GameController, so we know it will be called before our game gets under way. However, it is possible for users to start or stop their audio after the game has been initialized. The following section shows how we make sure our application is always in the correct state regarding the user’s choice of audio.
Responding to Other Audio Changes
Say the user starts the game without his own audio, plays for a while, and then decides to listen to his own audio. He might do this by hitting the Home button twice, swiping over to the audio controls, hitting Play, and then hitting Home twice again to return to the game. Therefore, we want to make sure we adjust the audio behavior of our application appropriately. Figure 11-1 shows an iPhone in the processing of doing just this.
Figure 11-1. User Playing Other Audio from the Background Menu
In Figure 11-1, the user has double-clicked the Home button and is about to press Play and listen to the Beastie Boys. What you can’t see in this picture is that the audio of the game stopped as soon as the Background menu opened. The game also paused. Additionally, after the user hits Play and returns to the game, the game will resume, but no more sound will be produced by the game. The ON/OFF switch controls whether or not nonbackground sound effects will be played over the Beastie Boys, but more on that later.
To understand how the application responds to the Background menu’s opening and closing, look at the code in Listing 11-3.
Listing 11-3. UIApplicationDelegate Tasks from AppDelegate.m
- (void)applicationWillResignActive:(UIApplication *)application
{
[self.viewController applicationWillResignActive];
}
- (void)applicationDidBecomeActive:(UIApplication *)application
{
[self.viewController applicationDidBecomeActive];
}
In Listing 11-3, we see the tasks applicationWillResignActive and applicationDidBecomeActive of the class AppDelegate. These are called whenever the application resigns being active or just becomes active. We simply call a corresponding method on our GameController—in this case, referenced by this.viewController. Listing 11-4 shows how our GameController responds to these events.
Listing 11-4. applicationWillResignActive and applicationDidBecomeActive (GameController.m)
-(void)applicationWillResignActive{
[self setIsPaused:YES];
[backgroundPlayer pause];
}
-(void)applicationDidBecomeActive{
[self setIsPaused:NO];
[self initializeAudio];
}
In Listing 11-4, the task applicationWillResignActive simply pauses the game by calling setIsPaused on self and pause on the AVAudioPlayer backgroundPlayer. When the application becomes active again, we resume the game and then call initializeAudio from Listing 11-2. By calling initializeAudio again, we have the chance to recheck if other audio is playing or not, ensuring our game respects the user’s wishes regarding sound.
By following the simple recipe above, we can ensure that any game will behave in a correct way regarding audio. Run the example app on your iPhone and play with some of the different user scenarios outlined above. Plug in your headphones, hit the Home button twice, adjust the volume, and start up a podcast. It all should be working as expected and in accordance with Apple’s guidelines.
In the following section, we will look at the example application in more detail and see how all of this audio stuff is used by a game to make bleeps and bloops.
The sample application that accompanies this chapter is a simple demonstration showing how sound can be incorporated in a game. Figure 11-2 shows the example in action.
Figure 11-2. Sample 11 in Action
In Figure 11-2, we see four elements on our familiar space background. We see two power-ups on the screen. These are being generated off the right side of the screen and move to the left. We also see a spaceship on the left that will be hit by power-ups that happen to be created midway up the screen. When a power-up collides with the spaceship, a sound is produced indicating the user got the power-up. The power-ups also start blinking when they get about halfway across the screen. Each time they blink off, a quick little click sound is made. When you run this example, you will also hear a background sound track that plays continuously while the game is running. The switch on the upper left controls whether such sound effects, like the blinking sound, should be heard regardless of other audio being played. Obviously, we would not include a switch like this on the screen while your game is playing, but it might be a good option on a setting screen someplace, probably defaulting to OFF.
As mentioned, there is a background sound track that plays in addition to the sound effects. As you will see in the following sections, these two types of audio are handled differently to make the developer’s life a little easier.
The following sections will go over this example piece by piece, highlighting how sound effects are played
Like many of the examples in this game, we start by extending GameController to specify our game specific features—the business logic of the game, if you will. Listing 11-5 shows how we set up this example.
Listing 11-5. doSetup (ViewController.m)
-(BOOL)doSetup{
if ([super doSetup]){
[self setGameAreaSize:CGSizeMake(480, 320)];
[self setIsPaused: NO];
NSMutableArray* classes = [NSMutableArray new];
[classes addObject:[Powerup class]];
[self setSortedActorClasses:classes];
viper = [Viper viper:self];
[self addActor: viper];
[self prepareAudio: AUDIO_POWERUP_BLINK];
[self prepareAudio: AUDIO_GOT_POWERUP];
[self playBackgroundAudio: AUDIO_BC_THEME];
return YES;
}
return NO;
}
In Listing 11-5, we see our standard GameController setup code. We specify the size of the game area, indicate that we want to keep track of Powerup actors, and we create and add a Viper. The last few things we do in doSetup is to set up the audio we want to play. We call prepareAudio:, then pass in the NSString constants AUDIO_POWERUP_BLINK and AUDIO_GOT_POWER. We will look at the task prepareAudio: shortly. For completeness, let’s take a quick look at the three NSString constants that are defined in Listing 11-6.
Listing 11-6. Sample 11-Prefix.pch (partial)
#define AUDIO_POWERUP_BLINK @"powerup_blink"
#define AUDIO_GOT_POWERUP @"got_powerup"
#define AUDIO_BC_THEME @"bc_theme"
In Listing 11-6, we see that we have defined three constants. Each is simply a string referring to the name of an audio file included in the project, excluding the file extension. Figure 11-3 shows these files in the Xcode project.
Figure 11-3. Including Audio in an Xcode Project
In Figure 11-3, we see that, to include audio in a project, we simply add the files as we would any other resource. Also, note that they are all m4a files. iOS is capable of playing many types of audio files, but AAC-encoded files are recommended, which these are. In our example, the m4a extension is assumed. 1
Looking back at Listing 11-5, we see that we pass these constants to two different tasks, prepareAudio: and playBackgroundAudio:. The task prepareAudio: gets an audio file ready so that the audio is played promptly when a game event requires a sound effect. The task playBackgroundAudio: loops a single track forever to provide background music for a game. Let’s start with prepareAudio:, as shown on Listing 11-7.
Listing 11-7. prepareAudio: (GameController.m)
-(AVAudioPlayer*)prepareAudio:(NSString*)audioName{
if (audioNameToPlayer == nil){
audioNameToPlayer = [NSMutableDictionary new];
}
NSURL *url = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:audioName ofType:@"m4a"]];
AVAudioPlayer* player = [[AVAudioPlayer alloc] initWithContentsOfURL:url error: nil];
[player prepareToPlay];
[audioNameToPlayer setValue:player forKey: audioName];
return player;
}
In Listing 11-7, we see the task prepareAudio:. This task is responsible for creating an AVAudioPlayer for each audio file passed in. We do this by first lazily initializing an NSMutableDictionary called audioNameToPlayer, which maps the NSString audioName to an AVAudioPlayer once constructed. To create an NSAudioPlayer, we first create an NSURL pointing to the desired m4a file in the project. To construct the NSAudioPlayer, we simply call initWithContentsOfURL:error:, ignoring any error.
We call prepareToPlay on the newly created player in order to make sure that the file is loaded and ready to make sound as soon as we call play on it sometime later.
The other sound-related task called in our doSetup task, from Listing 11-5, is playBackgroundAudio:, which is shown in Listing 11-8.
Listing 11-8. playBackgroundAudio: (GameController.m)
-(void)playBackgroundAudio:(NSString*)audioName{
if (backgroundPlayer != nil){
[backgroundPlayer stop];
}
backgroundPlayer = [self prepareAudio:audioName];
[backgroundPlayer setNumberOfLoops:-1];
if (!otherAudioIsPlaying){
[backgroundPlayer play];
}
}
In Listing 11-8, the task playBackgroundAudio: takes the name of an audio file. If an existing AVAudioPlayer backgroundPlayer exits, we stop it. Next, we call prepareAudio: to construct and initialize an AVAudioPlayer. To make the audio play indefinitely, we call setNumberOfLoops: and specify −1. While we want the background audio to play continuously, we can instead use setNumberOfLoops: to play an audio file a fixed number of times. The naming of the task setNumberOfLoops: is a little weird; it indicates the number of extra times a track is played. So, for example, if you specified 0, the audio would play once (the default number of times). If you specified 3, the audio track would play 4 times. Basically, setNumberOfLoops: specifies the number of additional times the audio should be played.
The last thing to do in playBackgroundAudio: is check if there is no other audio playing and start the audio by calling play on backgroundPlayer.
We have looked at how to specify what audio we will want to be able to play during the life of our game. We have also looked at how to set up a background audio track that plays indefinitely and with regard to other audio being played. Next, we will see how we play sound effects that are driven by in-game events.
Sound Effects for In-game Events
At this point, we have done a lot of setup to make sure our application plays audio correctly. We have also done a little work to make it easy for our game-related classes to play sound effects. In this section, we will look at some game logic and see how we play sound effects driven by an in-game event.
To review, in the chapter’s example, we have a number of power-ups that travel across the screen. When one of these power-ups collides with the spaceship, we play a sound effect. The logic that drives this behavior is found in Listing 11-9.
Listing 11-9. applyGameLogic (ViewController.m)
-(void)applyGameLogic{
if (self.stepNumber%120==0){
[self addActor:[Powerup powerup:self]];
}
for (Powerup* powerup in [self actorsOfType: [Powerup class]]){
if ([powerup overlapsWith:viper]){
[self removeActor: powerup];
[self playAudio: AUDIO_GOT_POWERUP];
}
}
}
In Listing 11-9, we see the task applyGameLogic. This task is called once per game step, giving subclasses of GameController a place to specify game-specific logic. In our case, we have things to do in this task: create Powerup actors and check collisions between Powerups and the spaceship.
We create a new Powerup every 120 steps in the game, or every 3 seconds (120 steps at 60 steps per second, 120/60 = 3). The newly created Powerup is simply added to the scene with the task addActor:. The logic that drives where the Powerup is created and how it moves is covered in more detail in Chapter 12.
The next step in applyGameLogic is to test to see if a Powerup has collided with the spaceship viper. We do this by iterating over each Powerup and testing if it collides with the task overlapsWith:. If we have a collision, we remove the Powerup and call playAudio:, passing in the NSString AUDIO_GOT_POWER.
The task playAudio: is shown in Listing 11-10.
Listing 11-10. playAudio: (GameController.m)
-(void)playAudio:(NSString*)audioName{
[self playAudio:audioName volume:1.0 pan:0.0];
}
-(void)playAudio:(NSString*)audioName volume:(float)volume pan:(float)pan{
if (alwaysPlayAudioEffects || !otherAudioIsPlaying){
AVAudioPlayer* player = [audioNameToPlayer valueForKey:audioName];
NSArray* params = [NSArray arrayWithObjects:player, [NSNumber numberWithFloat:volume], [NSNumber numberWithFloat:pan], nil];
[self performSelectorInBackground:@selector(playAudioInBackground:) withObject:params];
}
}
In Listing 11-10, we see the task playAudio:, which actually calls playAudio:volume:pan: with some default values. Actors or other game elements can call either version, depending on their needs. In playAudio:volume:pan:, we can see that we first check if we should actually play a sound at all. We do this by testing alwaysPlayAudioEffects and otherAudioIsPlaying. The BOOL alwaysPlayAudioEffects is a property of GameController and indicates if sound effects should ignore the fact that the user is playing his own audio. In this example, the value of alwaysPlayAudioEffects is controlled by the switch on the game screen. Flip the switch to experiment with the two different behaviors. The BOOL otherAudioIsPlaying is set in initializeAudio from Listing 11-2.
If it is determined that we should play a sound, playAudio:volume:pan: continues by finding the AVAudioPlayer in the NSMutableDictionary audioNameToPlayer associated with audioName. Once we have the player, we package player up with the volume and pan parameters in an NSArray called params. We then call performSelectorInBackground:withObject:, passing in the selector playAudioInBackground: and params.
The task playAudioInBackground: has nothing to do with background music; the word background indicates that this task should be performed in a background thread, hence the call to performSelectorInBackground:withObject:. Listing 11-11 shows the implementation of playAudioInBackground:.
Listing 11-11. playAudioInBackground: (GameController.m)
-(void)playAudioInBackground:(NSArray*)params{
AVAudioPlayer* player = [params objectAtIndex:0];
NSNumber* volume = [params objectAtIndex:1];
NSNumber* pan = [params objectAtIndex:2];
[player setCurrentTime:0];
[player setVolume: [volume floatValue]];
[player setPan: [pan floatValue]];
[player play];
}
In Listing 11-11, the task playAudioInBackground: is responsible for actually configuring the AVAudioPlayer and finally calling play on it. As mentioned, this task is called in a background thread. This is done because it greatly improves the performance of the application. If this were called on the same thread the game is running in, you would see considerable jerkiness in the game’s animations.
After pulling the player, volume, and pan objects out of the NSArray params, we prepare the player to play the audio. We set the currentTime property to 0, the start of the track; we also set the volume and pan values to be used. The volume value is from silent (0.0) to full volume (1.0). By full volume, we mean the maximum volume given the user’s volume settings. The pan value is like the balance knob on a stereo. The value −1.0 indicates all the way to the left, the value 0.0 indicates evenly left and right, and 1.0 indicates all the way to the right. (We will use this feature when we look at the Powerup’s blinking behavior.)
Lastly, we call play on player, which caused the sound to be emitted by the iOS device. Notice here that we can only have one AVAudioPlayer per type of sound—meaning that it is possible for a given AVAudioPlayer to be part way through playing when it is called to start over. This is a bit of a compromise in the design. On one hand, we are only consuming the memory of one AVAudioPlayer (and associated audio file) per type of sound effect. On the other hand, we will never hear two instances of the same sound mix together through the course of play. Further, a longer sound effect will be chopped off and replayed. This setup is probably good enough for most games, but in higher-end games this might not do.
We could imagine a setup where we have some pool of AVAudioPlayers for each type of sound, and the GameController would be responsible for figuring out which (if any) AVAudioPlayer is available to play a given sound effect. I think, however, that this level of complexity will require a lot of application-specific logic. For example, the requirements for each type of sound, and the rules governing what to do when an AVAudioPlayer is unavailable, will vary greatly in different games and for different sounds. I leave it up to the reader to extend GameController if this is a required feature for a game. Let me know what you come up with! You can email me at [email protected].
In our example, we have considered the simplest example of playing. We looked at playing a sound when a Powerup collides with the spaceship; this sound effect was called by the ViewController. One of the overarching design decisions being presented in this book is that actors should encapsulate as much logic as they can, being free agents in the scene to do as they wish, and this includes playing sounds. There is nothing interesting from a technical perspective about the following example, but it will illustrate how sound effects can be driven by a particular actor’s state. Let’s explore the sound that occurs when a Powerup blinks; this is shown in Listing 11-12.
Listing 11-12. step: (Powerup.m)
-(void)step:(GameController*)gameController{
if (self.center.x < −self.radius){
[gameController removeActor:self];
return;
}
if (self.center.x < gameController.gameAreaSize.width-100){
if ([gameController stepNumber] % 25 == 0){
if (self.state == STATE_GLOW){
float pan = 1.0 - (self.center.x / gameController.gameAreaSize.width)*2.0;
[gameController playAudio: AUDIO_POWERUP_BLINK volume:.1 pan: pan];
self.state = STATE_NO_GLOW;
} else {
self.state = STATE_GLOW;
}
}
}
}
In Listing 11-12, we see the step: task for the class Powerup. This task is called once per step in the game and is a place where an actor can specify some of its own behavior. The first test in this task simply checks if the Powerup is off the left side of the screen and removes itself accordingly. The second test checks to see if the Powerup should start blinking; if so, every 25 steps it alternates between STATE_GLOW and STATE_NO_GLOW. If the Powerup has just switched to STATE_NO_GLOW, it plays the audio AUDIO_POWERUP_BLINK. Notice how the pan for this audio is based on the x value of the Powerup. This makes it sound like the blinks are coming from one side of the screen or the other. To observe this effect, play the example code with headphones on.
Summary
In this chapter, we reviewed how to set up a game to be a good iOS citizen by responding correctly to the user’s behaviors and expectations. We also looked at some framework code in GameController to make playing audio easier for our particular games. We looked at differentiating background music from sound effects. Lastly, we looked at how we use this framework by exploring a simple example that played two different sound effects driven by different events in the game.
1There is a lot more to know about audio formats and encodings. Here is a good starting place to learn about Apple’s Core Audio Format: http://en.wikipedia.org/wiki/Core_Audio_Format.