Chapter 31. Trombone

Trombone is a much more sophisticated musical instrument app than the preceding chapter’s Cowbell app. You can move the slide up and down to different positions to play any note. (Other than starting at F, the slide positions bear little resemblance to real trombone slide positions!) This app supports two different sliding modes. If you use the left side of the screen, you can freely move the slide. If you use the right side of the screen, the slide snaps to the closest note line. Besides being an easier way to play this instrument, this means you could also use this app as a pitch pipe.

This trombone can play its notes in two octaves; to raise the sound by an octave, place a second finger anywhere on the screen. The most fun part about this app is that, like with a real trombone, you must actually blow on your phone to produce sound!

These last two app features require phone features discussed in later chapters (multi-touch and using the microphone) so that portion of the code is not explained in this chapter. Instead, the focus is on manipulating a single sound effect’s pitch and duration to create all the audio needed by this app.

The User Interface

Trombone has a main page, an instructions page, and a settings page. The code for the settings page is not shown in this chapter because, except for its page title, it is identical to the settings page shown in Chapter 34, “Bubble Blower.” It enables the user to calibrate the microphone in case producing sounds is too hard or too easy. The code for the instructions page is not shown either because it’s not interesting.

The main page, pictured in Figure 31.1 in its initial state, contains the moveable trombone slide, note guide lines, and buttons that link to the other two pages. Listing 31.1 contains the XAML.

Figure 31.1 The main page simulates the appearance of a real trombone.

Listing 31.1 MainPage.xaml—The User Interface for Trombone’s Main Page

<phone:PhoneApplicationPage x:Class="WindowsPhoneApp.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:phone="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"
    SupportedOrientations="Portrait">
  <Canvas x:Name="LayoutRoot">

    <!-- The stationary inner slide -->
    <Image Canvas.Left="72" Source="Images/innerSlide.png"/>
    <!-- The moveable outer slide -->
    <Image x:Name="SlideImage" Canvas.Left="72" Canvas.ZIndex="1"
           Source="Images/outerSlide.png"/>
    <!-- An instructions button -->
    <Rectangle Canvas.Left="18" Canvas.Top="30" Canvas.ZIndex="1"
               Width="48" Height="48" Fill="{StaticResource PhoneForegroundBrush}"
               MouseLeftButtonUp="InstructionsButton_Click">
      <Rectangle.OpacityMask>
        <ImageBrush ImageSource="/Shared/Images/normal.instructions.png"/>
      </Rectangle.OpacityMask>
    </Rectangle>

    <!-- A settings button -->
    <Rectangle Canvas.Left="18" Canvas.Top="94" Canvas.ZIndex="1"
               Width="48" Height="48" Fill="{StaticResource PhoneForegroundBrush}"
               MouseLeftButtonUp="SettingsButton_Click">
      <Rectangle.OpacityMask>
        <ImageBrush ImageSource="/Shared/Images/normal.settings.png"/>
      </Rectangle.OpacityMask>
    </Rectangle>
  </Canvas>
</phone:PhoneApplicationPage>

Notes:

→ The note guide lines pictured in Figure 31.1 are added in this page’s code-behind.

→ An application bar would get in the way of this user interface, so two rectangles acting as buttons are used instead. They use the familiar opacity mask trick to ensure they appear as expected for any theme.

→ The trombone slide consists of two images, one on top of the other. These two images are shown in Figure 31.2.

Figure 31.2 The slide consists of a moving image on top of a stationary image.

The Code-Behind

Listing 31.2 contains the code-behind for the main page.

Listing 31.2 MainPage.xaml.cs—The Code-Behind for Trombone’s Main Page

using System;
using System.Linq;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Navigation;
using System.Windows.Resources;
using System.Windows.Shapes;
using Microsoft.Phone.Controls;
using Microsoft.Xna.Framework.Audio; // For SoundEffect

namespace WindowsPhoneApp
{
  public partial class MainPage : PhoneApplicationPage
  {
    // The single sound effect instance
    SoundEffectInstance soundEffectInstance;

    string[] notes = { "G", "G",   "A", "A",   "B", "B",
                       "C",   "D", "D",   "E", "E",   "F" };

    // The relative distance of each note's pitch,
    // where 0 is the initial F and -1 is one octave lower
    double[] pitches = { -.9 /*G*/, -.82 /*G*/, -.75 /*A*/, -.68 /*A*/,
                         -.6 /*B*/, -.5 /*B*/, -.4 /*C*/, -.35 /*D*/,
                         -.25 /*D*/, -.18 /*E*/, -.08 /*E*/, 0 /*F*/ };

    // For microphone processing
    byte[] buffer;
    int currentVolume;

    // For several calculations
    const int TOP_NOTE_POSITION = 20;
    const int BOTTOM_NOTE_POSITION = 780;
    const int OCTAVE_RANGE = 844;

    public MainPage()
    {
      InitializeComponent();

      // Load the single sound file used by this app: the sound of F
      StreamResourceInfo info = App.GetResourceStream(
        new Uri("Audio/F.wav", UriKind.Relative));
      SoundEffect effect = SoundEffect.FromStream(info.Stream);

      // Enables manipulation of the sound effect while it plays
      this.soundEffectInstance = effect.CreateInstance();

      // The source .wav file has a loop region, so exploit it
      this.soundEffectInstance.IsLooped = true;

      // Add each of the note guide lines
      for (int i = 0; i < this.pitches.Length; i++)
      {
        double position = BOTTOM_NOTE_POSITION + this.pitches[i] * OCTAVE_RANGE;

        // Add a line at the right position
        Line line = new Line { X2 = 410,
          Stroke = Application.Current.Resources["PhoneAccentBrush"] as Brush,
          StrokeThickness = 5, Opacity = .8 };
        Canvas.SetTop(line, position);
        this.LayoutRoot.Children.Add(line);

        // Add the note label next to the line
        TextBlock label = new TextBlock {
          Text = this.notes[i][0].ToString(), // Ignore the , use 0th char only
          Foreground = Application.Current.Resources["PhoneAccentBrush"] as Brush,
          FontSize = 40 };
        Canvas.SetLeft(label, line.X2 + 12);
        Canvas.SetTop(label, position - 20);
        this.LayoutRoot.Children.Add(label);

        // Add the separately, simulating a superscript so it looks better
        if (this.notes[i].EndsWith(""))
        {
          TextBlock flat = new TextBlock { Text = "", FontSize = 25,
            FontWeight = FontWeights.Bold, Foreground =
            Application.Current.Resources["PhoneAccentBrush"] as Brush };
          Canvas.SetLeft(flat, line.X2 + label.ActualWidth + 6);
          Canvas.SetTop(flat, position - 21);
          this.LayoutRoot.Children.Add(flat);
        }
      }

      // Configure the microphone
      Microphone.Default.BufferDuration = TimeSpan.FromSeconds(.1);
      Microphone.Default.BufferReady += Microphone_BufferReady;
      // Initialize the buffer for holding microphone data
      int size = Microphone.Default.GetSampleSizeInBytes(
        Microphone.Default.BufferDuration);
      buffer = new byte[size];

      // Start listening
      Microphone.Default.Start();

      CompositionTarget.Rendering += delegate(object sender, EventArgs e)
      {
        // Required for XNA Sound Effect API to work
        Microsoft.Xna.Framework.FrameworkDispatcher.Update();

        // Play the sound whenever the blowing into the microphone is loud enough
        if (this.currentVolume > Settings.VolumeThreshold.Value)
        {
          if (soundEffectInstance.State != SoundState.Playing)
            soundEffectInstance.Play();
        }
        else if (soundEffectInstance.State == SoundState.Playing)
        {
          // Rather than stopping immediately, the "false" makes the sound break
          // out of the loop region and play the remainder
          soundEffectInstance.Stop(false);
        }
      };

      // Call also once at the beginning
      Microsoft.Xna.Framework.FrameworkDispatcher.Update();
    }

    protected override void OnNavigatedTo(NavigationEventArgs e)
    {
      base.OnNavigatedTo(e);

      // Subscribe to the touch/multi-touch event.
      // This is application-wide, so only do this when on this page.
      Touch.FrameReported += Touch_FrameReported;
    }

    protected override void OnNavigatedFrom(NavigationEventArgs e)
    {
      base.OnNavigatedFrom(e);
      // Unsubscribe from this application-wide event
      Touch.FrameReported -= Touch_FrameReported;
    }

    void Touch_FrameReported(object sender, TouchFrameEventArgs e)
    {
      TouchPoint touchPoint = e.GetPrimaryTouchPoint(this);

      if (touchPoint != null)
      {
        // Get the Y position of the primary finger
        double position = touchPoint.Position.Y;

        // If the finger is on the right side of the screen, snap to the
        // closest note
        if (touchPoint.Position.X > this.ActualWidth / 2)
        {
          // Search for the current offset, expressed as a negative value from
          // 0-1, in the pitches array.
          double percentage = (-BOTTOM_NOTE_POSITION + position) / OCTAVE_RANGE;
          int index = Array.BinarySearch<double>(this.pitches, percentage);
          if (index < 0)
          {
            // An exact match wasn't found (which should almost always be the
            // case), so BinarySearch has returned a negative number that is the
            // bitwise complement of the index of the next value that is larger
            // than percentage (or the array length if there's no larger value).
            index = ~index;

            if (index < this.pitches.Length)
            {
              // Don't always use the index of the larger value. Also check the
              // closest smallest value (if there is one) and snap to it instead
              // if it's closer to the current value.
              if (index > 0 &&
                  Math.Abs(percentage - this.pitches[index]) >
                  Math.Abs(percentage - this.pitches[index - 1]))
                  index--;

              // Snap the position to the new location, expressed in pixels
              position = BOTTOM_NOTE_POSITION +
                         this.pitches[index] * OCTAVE_RANGE;
            }
          }
        }

        // Place the outer slide to match the finger position or snapped position
        Canvas.SetTop(this.SlideImage, position - this.ActualHeight - 40);

        // See how many fingers are in contact with the screen
        int numPoints =
          (from p in e.GetTouchPoints(this)
           where
             p.Action != TouchAction.Up
           select p).Count();

        // 1 represents one octave higher (-1 represents one octave lower)
        int startingPitch = (numPoints > 1) ? 1 : 0;

        // Express the position as a delta from the bottom position, and
        // clamp it to the valid range. This gives a little margin on both
        // ends of the screen because it can be difficult for the user to move
        // the slide all the way to either end.
        double offset = BOTTOM_NOTE_POSITION -
          Math.Max(TOP_NOTE_POSITION, Math.Min(BOTTOM_NOTE_POSITION,
          touchPoint.Position.Y));

        // Whether it's currently playing or not, change the sound's pitch based
        // on the current slide position and whether the octave has been raised
        this.soundEffectInstance.Pitch =
          (float)(startingPitch - (offset / OCTAVE_RANGE));
      }

    }

    void Microphone_BufferReady(object sender, EventArgs e)
    {
      int size = Microphone.Default.GetData(buffer);
      if (size > 0)
        this.currentVolume = GetAverageVolume(size);
    }

    // Returns the average value among all the values in the buffer
    int GetAverageVolume(int numBytes)
    {
      long total = 0;
      // Although buffer is an array of bytes, we want to examine each
      // 2-byte value.
      // [SampleDuration for 1 sec (32000) / SampleRate (16000) = 2 bytes]
      // Therefore, we iterate through the array 2 bytes at a time.
      for (int i = 0; i < numBytes; i += 2)
      {
        // Cast from short to int to prevent -32768 from overflowing Math.Abs:
        int value = Math.Abs((int)BitConverter.ToInt16(buffer, i));
        total += value;
      }
      return (int)(total / (numBytes / 2));
    }

    // Button handlers

    void SettingsButton_Click(object sender, MouseButtonEventArgs e)
    {
      this.NavigationService.Navigate(
        new Uri("/SettingsPage.xaml", UriKind.Relative));
    }

    void InstructionsButton_Click(object sender, MouseButtonEventArgs e)
    {
      this.NavigationService.Navigate(
        new Uri("/InstructionsPage.xaml", UriKind.Relative));
    }
  }
}

Notes:

→ The single sound file used by this app is a recording of an F being played on a trombone. The different notes are created by dynamically altering the pitch of the F as it plays.

→ Rather than directly use the SoundEffect object, as in the preceding chapter, this app calls its CreateInstance method to get a SoundEffectInstance object. SoundEffectInstance provides a few more features compared to SoundEffect and, because it is tied to a single instance of the sound, it enables manipulation of the sound after it has already started to play. Trombone requires SoundEffectInstance for its looping behavior and its ability to modify the pitch of an already-playing sound.

→ SoundEffectInstance exposes an IsLooped property (false by default) that enables you to loop the audio indefinitely until Stop is called. This can behave in one of two ways, depending on the source audio file:

→ For a plain audio file, the looping applies to the entire duration, so the sound will seamlessly restart from the beginning each time it reaches the end.

→ For an audio file with a loop region, the sound will play from the beginning the first time through, but then only the loop region will loop indefinitely. Calling the default overload of Stop stops the sound immediately, but calling an overload and passing false for its immediate parameter finishes the current iteration of the loop, and then breaks out of the loop and plays the remainder of the sound.

Figure 31.3 demonstrates these two different behaviors. The latter behavior is perfect for this app, because it enables a realistic-sounding trombone note of any length, complete with a beginning and end that makes a smooth transition to and from silence. Therefore, the F.wav sound file included with this app defines a loop region. Although the sound file is less than a third of a second long, the loop region enables it to last for as long as the user can sustain his or her blowing.

Figure 31.3 Options for looping with SoundEffectInstance.IsLooped set to true.

→ In the CompositionTarget.Rendering event handler, the current volume from the microphone is continually compared against a threshold setting (adjustable on the settings page). If it’s loud enough and the sound isn’t already playing, Play is called. (The State check isn’t strictly necessary because, unlike SoundEffect.Play, SoundEffectInstance.Play does nothing if the sound is already playing.) If the sound is playing and the volume is no longer loud enough, then Stop(false) is called to break out of the loop and play the end of the sound.

→ Inside Touch_FrameReported, which detects where the primary finger is in contact with the screen and whether a second finger is touching the screen (as discussed in Part VII, “Touch and Multi-Touch”), the sound’s pitch is adjusted. The startingPitch variable tracks which octave the base F note is in (0 for the natural octave or 1 for an octave higher); then the distance that the finger is from the bottom of the screen determines how much lower the pitch is adjusted. As you can see from the values in the pitches array at the beginning of the listing, a D is produced by lowering the pitch of the F by 25% (producing a value of -.25 or .75 depending on the octave), and a B is produced by lowering the pitch of the F by half (producing a value of -.5 or .5 depending on the octave).

The Finished Product