Example 13-1: Play a Sample

The most common use of the Sound library is to play a sound as background music or when an event happens on screen. The following example builds on Example 8-5 to play a sound when the shape hits the edges of the screen. The blip.wav file is included in the media folder that can be downloaded by following the instructions in Chapter 7.

As with other media, the variable that will contain the SoundFile object is defined at the top of the sketch, it’s loaded within setup(), and after that, it can be used anywhere in the program:

add_library('sound')

blip = None
radius = 120
x = 0
speed = 1.0
direction = 1

def setup():
  global blip, x
  size(440, 440)
  ellipseMode(RADIUS)
  blip = SoundFile(this, "blip.wav")
  x = width/2 # Start in the center

def draw():
  global x, direction
  background(0)
  x += speed * direction
  if x > width-radius or x < radius:
    direction = -direction # Flip direction
    blip.play()
  if direction == 1:
    arc(x, 220, radius, radius, 0.52, 5.76) # Face right
  else:
    arc(x, 220, radius, radius, 3.67, 8.9) # Face left

The sound is triggered each time its play() method is run. This example works well because the sound is only played when the value of the x variable is at the edges of the screen. If the sound were played each time through draw(), the sound would restart 60 times each second and wouldn’t have time to finish playing. The result is a rapid clipping sound. To play a longer sample while a program runs, call the play() or loop() method for that sound inside setup() so the sound is triggered only a single time.

Example 13-2: Listen to a Microphone

In addition to playing a sound, Processing can listen. If your computer has a microphone, the Sound Library can read live audio through it. Sounds from the mic can be analyzed, modified, and played:

add_library('sound')

mic = None
amp = None

def setup():
  global mic, amp
  size(440, 440)
  background(0)
  # Create an audio input and start it
  mic = AudioIn(this, 0)
  mic.start()
  # Create a new amplitude analyzer and patch into input
  amp = Amplitude(this)
  amp.input(mic)

def draw():
  # Draw a background that fades to black
  noStroke()
  fill(26, 76, 102, 10)
  rect(0, 0, width, height)
  # The analyze() method returns values between 0 and 1, 
  # so map() is used to convert the values to larger numbers
  diameter = map(amp.analyze(), 0, 1, 10, width)
  # Draw the circle based on the volume
  fill(255)
  ellipse(width/2, height/2, diameter, diameter)

There are two parts to getting the amplitude (volume) from an attached microphone. The AudioIn class is used to get the signal data from the mic and the Amplitude class is used to measure the signal. Objects from both classes are defined at the top of the code and created inside setup().

After the Amplitude object (named amp here) is made, the AudioIn object (named mic) is patched in to the amp object with the input() method. After that, the analyze() method of the amp object can be run at any time to read the amplitude of the microphone data within the program. In this example, that is done each time through draw() and that value is then used to set the size of the circle.

In addition to playing a sound and analyzing sound as demonstrated in the last two examples, Processing can synthesize sound directly. The fundamentals of sound synthesis are waveforms that include the sine wave, triangle wave, and square wave.

A sine wave sounds smooth, a square wave is harsh, and a triangle wave is somewhere between. Each wave has a number of properties. The frequency, measured in hertz, determines the pitch—the highness or lowness of the tone. The amplitude of the wave determines the volume—the degree of loudness.

Example 13-3: Create a Sine Wave

In the following example, the value of mouseX determines the frequency of a sine wave. As the mouse moves left and right, the audible frequency and corresponding wave visualization increase and decrease:

add_library('sound')

sine = None
freq = 400.0

def setup():
  global sine
  size(440, 440)
  # Create and start the sine oscillator
  sine = SinOsc(this)
  sine.play()

def draw():
  background(176, 204, 176)
  # Map the mouseX value from 20Hz to 440Hz for frequency  
  hertz = map(mouseX, 0, width, 20.0, 440.0)
  sine.freq(hertz)
  # Draw a wave to visualize the frequency of the sound
  stroke(26, 76, 102)
  for x in range(width):
    angle = map(x, 0, width, 0, TWO_PI * hertz)
    sinValue = sin(angle) * 120
    line(x, 0, x, height/2 + sinValue)

The sine object, created from the SinOsc class, is defined at the top of the code and then created inside setup(). Like working with a sample, the wave needs to be played with the play() method to start generating the sound. Within draw(), the freq() method continuously sets the frequency of the waveform based on the left-right position of the mouse.

Example 13-4: Saving Images

This example shows how to save images by storing enough frames for a two-second animation. It loads and moves the robot file from “Robot 5: Media”. See Chapter 7 for instructions for downloading the file and adding it to the sketch. The example runs the program at 30 frames per second and then exits after 60 frames:

bot = None
x = 0

def setup():
  global bot
  size(720, 480)
  bot = loadShape("robot1.svg")
  frameRate(30)

def draw():
  global x
  background(0, 153, 204)
  translate(x, 0)
  shape(bot, 0, 80)
  saveFrame("frames/SaveExample-####.tif")
  x += 12
  if frameCount == 60: 
    exit()

Processing will write an image based on the file extension that you use (.png, .jpg, and .tif are all built in, and some platforms may support others). A .tif image is saved uncompressed, which is fast but takes up a lot of disk space. Both .png and .jpg will create smaller files, but because of the compression, will usually require more time to save, making the sketch run slowly.

If your desired output is vector graphics, you can write the output to PDF files for higher resolution. The PDF Export library makes it possible to write PDF files directly from a sketch. These vector graphics files can be scaled to any size without losing resolution, which makes them ideal for print output—from posters and banners to entire books.

Example 13-5: Draw to a PDF

This example builds on Example 13-4 to draw more robots, but it removes the motion. The PDF library is imported at the top of the sketch to extend Processing to be able to write PDF files.

This sketch creates a PDF file called Ex-13-5.pdf because of the third and fourth parameters to size():

add_library('pdf')

bot = None

def setup():
  global bot
  size(600, 800, PDF, "Ex-13-5.pdf")
  bot = loadShape("robot1.svg")

def draw():
  background(255)
  for i in range(100):
    rx = random(-bot.width, width)
    ry = random(-bot.height, height)
    shape(bot, rx, ry)
  exit()

The geometry is not drawn on the screen; it is written directly into the PDF file, which is saved into the sketch’s folder. The code in this example runs once and then exits at the end of draw(). The resulting output is shown in Figure 13-1.

There are more PDF Export examples included with the Processing software. Look in the PDF Export (pdf) section of the Processing examples to see more techniques.

Figure 13-1. PDF export from Example 13-5

Example 13-6: Read a Sensor

The following Arduino code is used with the next three Processing examples:

// Note: This is code for an Arduino board, not Processing

int sensorPin = 0;  // Select input pin
int val = 0;

void setup() {
  Serial.begin(9600);  // Open serial port
}

void loop() {
  val = analogRead(sensorPin) / 4;  // Read value from sensor
  Serial.write((byte)val);  // Print variable to serial port
  delay(100);  // Wait 100 milliseconds
}

There are two important details to note about this Arduino example. First, it requires attaching a sensor into the analog input on pin 0 on the Arduino board. You might use a light sensor (also called a photo resistor, photocell, or light-dependent resistor) or another analog resistor such as a thermistor (temperature-sensitive resistor), flex sensor, or pressure sensor (force-sensitive resistor). The circuit diagram and drawing of the breadboard with components are shown in Figure 13-2. Next, notice that the value returned by the analogRead() function is divided by 4 before it’s assigned to val. The values from analogRead() are between 0 and 1023, so we divide by 4 to convert them to the range of 0 to 255 so that the data can be sent in a single byte.

Figure 13-2. Attaching a light sensor (photo resistor) to analog in pin 0

Example 13-7: Read Data from the Serial Port

The first visualization example shows how to read the serial data in from the Arduino board and how to convert that data into the values that fit to the screen dimensions:

add_library('serial')
port = None  # for object from Serial class
val = 0.0    # Data received from the serial port

def setup():
  global port
  size(440, 220);
  # IMPORTANT NOTE:
  # The first serial port retrieved by Serial.list()
  # should be your Arduino. If not, uncomment the next
  # line by deleting the # before it. Run the sketch
  # again to see a list of serial ports. Then, change
  # the 0 in between [ and ] to the number of the port
  # that your Arduino is connected to.
  #print Serial.list()
  arduinoPort = Serial.list()[0]
  port = Serial(this, arduinoPort, 9600)

def draw():
  global val
  if port.available() > 0:       # If data is available,
    val = port.read()            # read it and store it in val
    val = map(val, 0, 255, 0, height) # Convert the value
  rect(40, val-10, 360, 20)

The Serial library is imported on the first line and the serial port is opened in setup(). It may or may not be easy to get your Processing sketch to talk with the Arduino board; it depends on your hardware setup. There is often more than one device that the Processing sketch might try to communicate with. If the code doesn’t work the first time, read the comment in setup() carefully and follow the instructions.

Within draw(), the value is brought into the program with the read() method of the Serial object. The program reads the data from the serial port only when a new byte is available. The available() method checks to see if a new byte is ready and returns the number of bytes available. This program is written so that a single new byte will be read each time through draw(). The map() function converts the incoming value from its initial range from 0 to 255 to a range from 0 to the height of the screen; in this program, it’s from 0 to 220.

Example 13-8: Visualizing the Data Stream

Now that the data is coming through, we’ll visualize it in a more interesting format. The values coming in directly from a sensor are often erratic, and it’s useful to smooth them out by averaging them. Here, we present the raw signal from the light sensor illustrated in the top half of the example and the smoothed signal in the bottom half:

add_library('serial')
port = None # for Serial object
val = 0.0   # Data received from the serial port
x = 0
easing = 0.05
easedVal = 0.0

def setup():
  global port
  size(440, 440)
  frameRate(30)
  arduinoPort = Serial.list()[0]
  port = Serial(this, arduinoPort, 9600)
  background(0)

def draw():
  global x, val, easedVal
  if port.available() > 0:      # If data is available,
    val = port.read()           # read it and store it in val
    val = map(val, 0, 255, 0, height) # Convert the values
  targetVal = val;
  easedVal += (targetVal - easedVal) * easing

  stroke(0)
  line(x, 0, x, height)            # Black line
  stroke(255)
  line(x+1, 0, x+1, height)        # White line
  line(x, 220, x, val)             # Raw value
  line(x, 440, x, easedVal + 220)  # Averaged value

  x += 1
  if x > width:
    x = 0

Similar to Example 5-8 and Example 5-9, this sketch uses the easing technique. Each new byte from the Arduino board is set as the target value, the difference between the current value and the target value is calculated, and the current value is moved closer to the target. Adjust the easing variable to affect the amount of smoothing applied to the incoming values.

Example 13-9: Another Way to Look at the Data

This example is inspired by radar display screens. The values are read in the same way from the Arduino board, but they are visualized in a circular pattern using the sin() and cos() functions introduced earlier in Example 8-12 to Example 8-15:

add_library('serial')

port = None  # Serial class object
val = 0.0    # Data received from the serial port
angle = 0.0
radius = 0.0

def setup():
  global port
  size(440, 440)
  frameRate(30)
  strokeWeight(2)
  arduinoPort = Serial.list()[0]
  port = Serial(this, arduinoPort, 9600)
  background(0)

def draw():
  global val, angle, radius
  if port.available() > 0:  # If data is available,
    val = port.read()       # read it and store it in val
    # Convert the values to set the radius
    radius = map(val, 0, 255, 0, height * 0.45)

  middleX = width/2
  middleY = height/2
  x = middleX + cos(angle) * height/2
  y = middleY + sin(angle) * height/2
  stroke(0)
  line(middleX, middleY, x, y)

  x = middleX + cos(angle) * radius
  y = middleY + sin(angle) * radius
  stroke(255)
  line(middleX, middleY, x, y)

  angle += 0.01

The angle variable is updated continuously to move the line drawing the current value around the circle, and the val variable scales the length of the moving line to set its distance from the center of the screen. After one time around the circle, the values begin to write on top of the previous data.

We’re excited about the potential of using Processing and Arduino together to bridge the world of software and electronics. Unlike the examples printed here, the communication can be bidirectional. Elements on screen can also affect what’s happening on the Arduino board. This means you can use a Processing program as an interface between your computer and motors, speakers, lights, cameras, sensors, and almost anything else that can be controlled with an electrical signal. Again, check out http://www.arduino.cc for more information about Arduino.