Adding sound to a game will add an additional dimension and can help bring the game come to life. This can be achieved by adding special effect sounds or adding background music to set the mood. You may also use the sound as a key component in the game.
As well as looking at how music can be added to a game through Pygame Zero, this chapter will also look at ways of creating the sound effects or music and some of the tools that can be used to process the sounds.
This chapter starts with looking at how you can create your own sounds and music. If you are just interested in using sound effects or music that have been created by someone else, you can skip to later in the chapter where the sounds are added to a Pygame Zero game.
Recording Sound Effects
For realistic sound effects, they are often created by recording real sounds. It may not however be possible to record the effect you are creating in the game. If you don’t happen to own a challenger tank, then you may need to look at something that sounds like a tank rather than recording a real tank. If you are creating a futuristic sci-fi game, then you may need to look at sounds being computer generated.
Even if you can record the exact effect that you want, that may not sound quite right for a game. One of the things I looked at was how you could create the sound of a steam train. I have several preservation railways within a reasonable distance, so I visited them to record the sounds. One problem is that there is a lot of additional background noise from people, pets, and other things around such as car traffic. Also, the sounds recorded while realistic did not match the sound that you may expect or that fit in with what is happening in the game. For example, when recording the sound of the train, the sound of the locomotive was accompanied by lots of different noises, such as the carriages clanging and the sound of the wheels squealing against the track. I found a better sound was achieved by recording the sound of the locomotive when it was uncoupled from the train rather than when it was pulling a train.
You probably won’t be wanting to carry a Raspberry Pi, screen, and accessories around when you want to record sounds. In that case you can use a portable recorder, perhaps a mobile phone either using a video recorder or using an audio recording tool. Details are provided on how you can convert and edit suitable audio formats using Audacity if you have captured them with a mobile phone.
Creating Artificial Sound Effects
The crunch of walking feet using a shoe in a tray of gravel.
The clip-clop of horse hooves by tapping coconut shells together.
Explosions based on fireworks. If local laws don’t permit consumer fireworks, then you could record a professional display.
Water sounds created in a bathtub.
I have used artificial sound effects in creating the sounds for the tank game. The sound of the tank firing is based on popping a balloon, with the time slowed down. The sound of the explosion was recorded at a public firework display.
You can also create sound effects synthetically using music creation tools such as Sonic Pi. It is possible to use different shaped waveforms and adding audio effects to create various sounds, particularly useful for sci-fi type effects.
EpicSound – www.epicsound.com/sfx/
The Art of Foley – www.marblehead.net/foley/specifics.html
Recording Audio on the Raspberry Pi
Raspberry Pi with USB microphone
Auto
Force 3.5mm (“headphone”) jack
Force HDMI
Connecting a USB Microphone
Partial output of dmesg showing USB microphone
This example is using a Fifine Technology USB microphone. It uses the driver Echo-058.
Raspberry Pi sound settings with USB microphone
Using arecord
Once the microphone is connected, then there are a few different tools that can be used to record sounds. For a simple command-line tool, arecord is included in the standard NOOBS image.
Output of arecord –l command
The card number (in this case 1) and the device number (in this case 0) form the basis of the device reference which in this case is hw:1,0. The plughw plugin needs to be used; in this case, the device is plughw:1,0.
An alternative to using the command line is the graphical application Audacity, which will be covered next.
Audacity
Audacity is a powerful tool which can be used for recording and editing audio. Here you will see how Audacity can be used for recording audio on the Raspberry Pi, converting audio formats, extracting audio from video files, and trimming audio files.
This will add an option to the “Sound & Video” menu. For other operating systems, you can download Audacity from www.audacityteam.org.
Screenshot of Audacity audio editor
Here are a few suggestions of things you may like to try which will help familiarize yourself with some of the features of Audacity.
Recording Sounds with Audacity
Launch Audacity and it will not show any sound waveforms.
Ensure that the microphone is selected as the input device (shown alongside the microphone icon).
Click the red record button and talk into the microphone or record nearby sounds.
Stop recording.
Export the audio as a suitable sound format (WAV and OGG are good formats for use in Pygame Zero).
Convert Audio Formats
Close any existing project.
Load an audio file using open from the file menu.
Choose export to save as a different audio format.
Extract Audio from Video Files
As well as reading audio files, Audacity can extract the audio from video format files such as MP4 and AVI. The process is the same as converting audio formats except that you select video as the source instead of an audio file.
Trim Audio Files
Open the audio file.
Select the part to be trimmed using the mouse along the waveform.
Press the Delete key.
Export the updated sound to a suitable file format.
This has covered some useful features but has only scratched the surface of what Audacity can do. It can handle multiple tracks and provides filters that allow you to apply different effects to the sounds.
Creating Music with Sonic Pi
There are multiple options for creating music. A useful tool that is included on the Raspberry Pi is Sonic Pi.
Screenshot of Sonic Pi music creation tool
The program has several buffer text edit tabs where code can be entered. The code is based on Ruby which is quite different from Python. It’s not possible to go into detail in this book, but an example will be given of how it can be used to create background music.
Code to create music in Sonic Pi
Enter the code into one of the buffers and press Run.
This code works by playing musical notes which are stored in an array (list), which is played in the loop. The tune is a simplified version of When the Saints Go Marching In. It’s a traditional song which doesn’t have any copyright issues.
Another musical tune created in Sonic Pi
This uses three different loops with some special effects. This creates a tune that could be used as a background music for a game.
To record the music as a WAV file that can be used in Pygame Zero, click the record button before starting the music, then click the record button again to stop recording, and save it as a file. You will then need to trim out any unwanted silence at the beginning or end using Audacity.
The code is based on Ruby which is very different from Python and is beyond the scope of this book. To learn more about Sonic Pi, there is a good tutorial included in the program. Look in the bottom left corner of Sonic Pi for more details.
Downloading Free Sounds and Music
There are many places where you can download free sounds and music. These include recordings of live effects as well as original music which is made available for free use. Whenever you get sound or music from one of these sites, you need to check that the license allows for your intended use.
Two popular sites for sound effects are Sound Bible (http://soundbible.com/) and Freesound (https://freesound.org). Most of the sound effects listed on the sites are under an Attribution license which means you can use for most purposes as long as you credit the creator. Some of the samples do restrict the sounds to personal use only, so you may need to be careful with those.
If you are looking for music, then there are several links on the Creative Commons web site http://bit.ly/ccmusic1. This site links to other web sites known to have free music, but you will need to check for any restrictions on use.
Adding Sound Effects in Pygame Zero
Having created or downloaded an appropriate sound effect, the next stage is to add it to your games. The sounds can be in WAV or OGG formats.
To play sounds in Pygame Zero, first create a new sub-directory called sounds and copy your sound effects in there. The format of the command to play the sound is sounds, followed by the filename (without any extension) and by the appropriate method such as play.
This method should only be used for short sound effects. It loads the entire sound file into memory and can have a significant performance impact if you try to use it on long music files. If you want to play longer pieces of music, then see “Playing Music in Pygame Zero” later in this chapter.
I have included two sound effects in the sounds sub-directory called tankfire.wav and explode.wav. These are used to add some sound effects to the tank game created in the last chapter.
This should be repeated for 'player2fire' and 'game_over_2'. All the required files are included in the supplied source code.
Playing Music in Pygame Zero
When you need some music to play longer, then there is a music player option. The built-in music object provides the ability to play music by loading the track a bit at a time. It only allows a single track to play at a time but can be combined with sounds to have special effects playing at the same time as background music. The music files should be stored in a directory called music .
This is a relatively new feature in Pygame Zero and comes with a warning. The music support depends upon the computer system and how well they support playback of a particular codec. It should work with MP3, OGG, and WAV files. MP3 music cannot be played on certain Linux systems, which may be due to patents that have now expired. There have also been reported issues with OGG files. It would seem that WAV may be the safer option, although that may be just that there have been less reports of issues. WAV files are uncompressed which can result in large file sizes.
In either case it will stop any previous track or any in the queue. If you would like to add another track to play next after the current one, then you can use music.queue.
It is possible to stop, pause, and unpause the music as well as changing the volume through set_volume prefixing the method name with the music object.
Piano Game Created with Tones
Another alternative with Pygame Zero is to play computer-generated sounds using the built-in tone generator. The tone generator can be a useful way for creating sounds, but it uses synthesized sounds and is not as good quality as could be created using sampled sounds. It was added in version 1.2 of Pygame Zero, which is included in the latest version of Raspbian and Mu. It may not work on some older versions.
Screenshot of Piano Game
Clicking any of the keys will play the appropriate note. Clicking the Demo button will play a demonstration of the tune. Clicking Start will play the game; clicking the correct key when the note reaches the target line will score a point.
This game is designed for use with the Raspberry Pi touch screen. It can still be used with a mouse but is harder to play when you need to move the mouse pointer. A limitation to this game is that the player can only press one key at a time. This is a limitation of Pygame Zero, which does not support multi-touch. If you wanted to use multi-touch, then you would need to look at a different programming framework such as Kivy, but that is beyond the scope of this book.
Code for Piano Game
I won’t go through this line by line, but I will go through some of the key parts of how the code works.
Starting from the top, you will see that the screen resolution is set to a HEIGHT of only 410. This is because of the resolution of the 7-inch screen after subtracting the top menu bar and window decoration.
The tune is an array which lists the notes that need to be played. In this case it is for When the Saints Go Marching In. The music originates from around the late 19th to early 20th century. You could replace that with a more modern tune, but in that case, you would need to take into consideration any copyright issues if you then redistributed the game. The tune needs to be quite simple as only one note can be played at a time, and it will only play quarter notes (crotchets) and rests. In this case the music has been simplified and altered slightly. Chords have been replaced with single notes and the sustain removed on longer notes. The tune should still be recognizable. The notes are stored in the list as strings which are based on the note and the octave, where C4 is middle C. If there is a sharp, then that can be indicated by adding an # between the note and the octave number.
There are several other variables and two Actors which represent the two buttons which are created as images. The tempo is determined by the number of beats_per_minute, which is then converted into the length of time between each beat, measured in seconds. In the case of 116 beats per minute, that is the number of quarter notes (crotchets) in a minute. This works out as 0.51 seconds between each quarter note which is each entry in the list. The update function is called approximately every 0.016 seconds, which should provide a reasonably accurate timing. The note duration is stored in the variable time_note, which is half of the time between notes so that the notes don’t merge if played quickly.
Another variable is the note_position which is used to indicate the position of the array where the current note is. The variable starts at –10 because that allows the notes to fall from the top of the screen. Only when the note_position reaches 0 will that note be played (if playing the demo) or the player needs to click the note (in the game). After the variables are the functions followed by a call to setup. This is because the functions need to be loaded into memory before the setup function tries to use them. Even though the call to setup is the last line of the file, it is still run before Pygame Zero runs the update and draw functions.
The setup function creates the rect objects needed to create the keyboard and pre-loads all the notes of the keyboard. The keys are created as two separate lists, the accidentals (sharp and flat keys) are the black keys and the natural keys are white. The accidentals are referred to as sharp keys in the code as they are created offset from previous natural key, so the sharp named C3 is C#3.
The draw_piano function calls screen.draw.rect for the natural keys and screen.draw.filled_rect for the accidentals.
The on_mouse_down function handles the clicks on the buttons, which sets the state to demo or game as appropriate. It also detects if any of the keys on the piano keyboard are pressed, and if so, it starts the note playing. If in game mode it increases the score if the correct key being pressed.
The update function checks to see if enough time has expired for the next note. It uses the argument timer_interval which gives the amount of time that has passed since the update function was last run. It uses this to track the time since the last note was played. If it has not reached the time in time_crotchet, then it returns from the function. If the timer has exceeded that time, then it can update the note_position if it is in either the demo or game states.
The draw function displays the buttons, keyboard, and any notes or text that needs to be displayed. A line is drawn as the target for when the note should be played. This uses screen.draw.line which uses the start and end coordinates. It also displays the score during the game and the game over message when complete.
This is a simple fun game but would need quite a lot more to create a game that could be used to help teach someone how to play the piano. As mentioned previously the lack of multi-touch is quite limiting. There are still things that you could do to improve the game, such as lighting up the keys when they should be pressed (by using filled_rect with an appropriate color) and providing a way to change the tempo. It’s also limited in playing only quarter notes, which could be changed but would involve loading multiple versions of each note depending upon the duration of the note.
Summary
This chapter has covered a few different ways of making and using sounds and music in Sonic Pi. This has included using the Raspberry Pi as a recording device or for converting and editing sounds recorded on another device. It has also covered creating your own music using Sonic Pi.
It then covered the three different ways of playing sounds through Pygame Zero. Sound effects played using the sound object, music played with the music object, and tone using the tone object.
The next chapter is on object-oriented programming, showing an alternative way to creating software using Python.