Have you ever wanted a snack or soda and accidently put foreign currency in your local vending machine? Or, have you ever accidently put a DVD into a CD player? What happened? Most likely, either nothing happened or some type of error was displayed. These scenarios show that devices are created with specific formats in mind. Likewise, the MediaElement expects certain formats.

The MediaElement supports a powerful array of audio and video formats that empower you to deliver high-quality media experiences over the internet. The accepted audio formats ensure a truly high-fidelity aural experience. At the same time, the supported video formats ensure a viewing experience that can scale from mobile devices all the way up to high-definition displays. Table 20.1 shows the formats supported by the MediaElement.

By targeting these media formats, the Silverlight runtime can be a self-contained environment for media experiences. Once your users install the Silverlight runtime, they can run all the supported media formats without having to download and install additional codecs.

The format for media is important, but the delivery method is equally so. Table 20.2 lists the delivery methods Silverlight recognizes for audio and video.

In addition to the progressive download formats, table 20.2 shows two different streaming methods: Smooth Streaming and Windows Media Streaming over HTTP.

Smooth Streaming is an HTTP-based multiple bit rate (MBR) adaptive media streaming service implemented on Internet Information Server (IIS) on Windows servers. Smooth Streaming dynamically detects client bandwidth and CPU usage and adapts to conditions in close to real-time. Smooth Streaming provides:

For example, if you're watching an HD video on your client and suddenly you start a CPU-intensive process such as a large compile, rather than drop frames, Smooth Streaming detects the condition and lowers the quality of the video (lowers the bit rate, which typically means a lower resolution) so your viewing sessions continues uninterrupted.

Similarly, if you're watching an HD video and someone in your house starts a large download, effectively taking up a large portion of your internet bandwidth, Smooth Streaming will adapt to that as well, lowering the bit rate to fit into the available bandwidth.

Finally, Smooth Streaming supports simplified caching of content, as the individual chunks are individual files, easily cached using standard HTTP file caching mechanisms. The caches need not know anything about media formats; the bits are just files. For the same reasons, proxies work just as well, requiring no special open ports or knowledge of the formats.

Smooth Streaming delivers small content fragments (about two to four seconds worth of video) to the client, and verifies (with the help of Silverlight) that the content all arrived on time and played at the expected quality level. If a fragment doesn't meet these requirements due to bandwidth or processor restrictions, the next fragment will be delivered at a lower quality level. If the conditions were favorable, the next fragment will be delivered at the same or higher quality level.

Similarly, if the video is available in 1080p HD, but the user is watching it on a display at 720p resolution, Smooth Streaming will send down only the 720p size chunks, saving bandwidth and processing time.

On the server, this requires that the videos be encoded to several different formats. IIS Smooth Streaming keeps all the chunks for a given format in a single MP4 file, but delivers the chunks as individual logical files. This makes server file management (and file access) easier, while still providing for caching of chunks by local proxies and downstream servers. Smooth Streaming files have the extension .ismv for video plus audio, and .isma for audio-only. Figure 20.1 shows the structure of the Smooth Streaming file on the server.

Figure 20.1. The Smooth Streaming server-side file format

The file includes a file type header to let us know this is the smooth streaming file. Next, it includes Movie Metadata (moov) that describes what the file contains. Following that are the individual two-second fragments for the entire movie. Each fragment contains header information for the fragment, as well as the fragment bits themselves. The file closes with an "mfra" index that allows for easy and accurate seeking within the file.^[1]

In addition to the media file described here, Smooth Streaming also uses a .ism manifest file for the server, which describes the relationships between the different server files, and a .ismc client manifest file, describing the available streams, codecs, bit rates, markets, and so on. This .ismc file is what's first delivered to the client when the video is requested.

An online example of Smooth Streaming with IIS and Silverlight may be seen on the IIS Smooth Streaming site here: http://www.iis.net/media/experiencesmoothstreaming. Other examples of Smooth Streaming through a CDN may be seen at http://www.smoothhd.com.

To encode video for use with Smooth Streaming, you use Microsoft Expression Encoder. Once the videos are encoded, you can use the Expression Encoder Smooth Streaming template to serve as the start of your video player, or you can use the Silverlight Media Framework, covered in section 20.5.

The Silverlight Media Framework (SMF) is the easiest way to incorporate Smooth Streaming into your application. Before we cover that, let's look at other forms of streaming and downloading available to us.

Though now generally out of favor due to the introduction of Smooth Streaming, Silverlight still supports streaming media content over HTTP through server-side play lists and the MMS protocol. The MMS protocol was built for sending many short messages to a client, and uses a URI that begins with mms:// instead of http:// or https://. When a media file is streamed through this protocol, your Silverlight application maintains an open connection with the hosting server. This has two advantages. It enables you to jump to any point in time within a media file, and streaming usually provides a more cost-effective approach for delivering audio and video content because only the requested content is downloaded, plus a little extra. This content is configurable through the BufferingTime property.

The BufferingTime property enables you to view or specify how much of a buffer should be downloaded. By default, this TimeSpan value is set to buffer 5 seconds worth of content. If you're streaming a 1-minute video, the video won't begin playing until at least 5 seconds of it has been retrieved. While this retrieval is occurring, the Current-State property of the MediaElement (which we'll discuss shortly) will be set to Buffering. While the MediaElement is in a Buffering state, it'll halt playback. You can check to see what percentage of the buffering is completed by checking the BufferingProgress property.

The BufferingProgress property gives you access to the percentage of the completed buffering. Because this property value is always between 0.0 and 1.0, you need to multiply it by 100 to get the percentage. When this property changes by a value greater than 5 percent, the BufferingProgressChanged event will be fired. This event gives you the flexibility to keep your users informed through a progress bar or some other UI construct. As you can imagine, this type of component can be valuable when you're streaming content.

Often, streamed content can be quite lengthy. Because of this, it can be advantageous to use MBR) files. MBR files enable you to provide the highest quality experience based on the available bandwidth. The really cool part is that the MediaElement will automatically choose which bit rate to use based on the available bandwidth. In addition, the MediaElement will automatically attempt to progressively download the content if it can't be streamed. That's thinking progressively.

Progressive downloading involves requesting a media file over the HTTP or HTTPS protocol. When this occurs, the requested content is temporarily downloaded to a user's computer, enabling the user to quickly access any part of the media that has been downloaded. In addition to fast access, using a progressive download generally provides a higher-quality media experience. Progressive downloading usually requires a longer initial wait time than streaming, so you may want to keep your users informed of how much wait time is left.

Keeping your users informed is made possible through two key items within the MediaElement. The first item is a property called DownloadProgress. It gives you access to the percentage of the content that has been downloaded. The other item is an event called DownloadProgressChanged. This event gives you the ability to do something such as update a progress bar whenever the DownloadProgress property changes. In listing 20.1, both these items are used to show the percentage of requested content that's available.

Example 20.1. The percentage of content ready for use within a MediaElement

This example shows a large video file (~13MB) being progressively downloaded . As this download progresses, the completion percentage is calculated . This percentage is then formatted and presented to the user as the video is downloaded.

Whether you stream content or progressively download it, the MediaElement expects certain formats. These file formats are then retrieved over one of the accepted protocols (HTTP, HTTPS, or MMS). The Source property simplifies this retrieval process, and it works with both audio and video files. Once the media source is loaded, the MediaElement can be used to configure the playback of a media item or obtain status information. These items are available through a set of commonly used properties.

The AutoPlay property specifies whether the MediaElement will automatically begin playing. By default, a MediaElement will begin playing as soon as the content referenced in the Source property is loaded. You can disable this default behavior by changing the AutoPlay bool property to false. As you can imagine, once a media file has begun playing, there may be times when you want to be able to pause it.

Sometimes you may want to allow a user to halt the playback of a MediaElement. By default, the MediaElement will allow you to do this. But, by setting the CanPause property of the MediaElement to false, you can prevent your users from pausing the playback. If you allow the pausing function and a user decides to halt the playback, it'll change the value of the CurrentState property.

The CurrentState property represents the mode the MediaElement is in. This mode is exposed as a value of the System.Windows.Media.MediaElementState enumeration. This enumeration provides all the possible states a MediaElement can be in. These states are listed and described in table 20.3.

Table 20.3 shows the options available within the MediaElementState enumeration. This enumeration is used by the read-only CurrentState property. Considering that this property is read-only, how does it get set? This property is altered through a variety of methods you'll learn about later in this chapter. Anytime the Current-State property value is changed, an event called CurrentStateChanged is fired. The state of the media item is a natural part of working with the MediaElement, as is the duration.

The NaturalDuration property gives you access to the natural duration of a media item. This duration is available once the MediaElement has successfully opened a media stream, so you shouldn't use the NaturalDuration property until the MediaOpened event has fired. Once the MediaOpened event has fired, you can access the total length of a media item, as shown here:

void me_MediaOpened(object sender, RoutedEventArgs e)
{
  tb.Text = "Your video is " + me.NaturalDuration + " long.";
}

This example displays the total length of a media item in an assumed TextBlock. This task takes place when the MediaOpened event of a MediaElement has triggered, so you can assume that the media stream has been successfully accessed. Then, you use the NaturalDuration property to show the length of the media stream. This length is stored as a TimeSpan within the NaturalDuration property.

The NaturalDuration property is a System.Windows.Duration entity. This type of entity is a core element of the .NET Framework, and it exposes a property called Has-TimeSpan that signals whether a TimeSpan is available. In the case of a MediaElement, this property value will always be true, enabling you to access highly detailed information about the length of a media stream through the TimeSpan property. This property is demonstrated in this example:

void me_MediaOpened(object sender, RoutedEventArgs e)
{
  StringBuilder sb = new StringBuilder();
  sb.Append("Your video is ");
  sb.Append(me.NaturalDuration.TimeSpan.Minutes);
  sb.Append(" minutes, ");
  sb.Append(me.NaturalDuration.TimeSpan.Seconds);
  sb.Append(" seconds, and ");
  sb.Append(me.NaturalDuration.TimeSpan.Milliseconds);
  sb.Append("milliseconds.");
  tb.Text = sb.ToString();
}

This shows how to access detailed information about the length of a media item. As you probably know, this information, as well as the position of the playback, is part of almost any online media player.

The Position property represents a point, or location, within a MediaElement. This value can be read regardless of the CurrentState of the MediaElement, and it can be set if the MediaElement object's CanSeek property is true.

The CanSeek property determines whether the Position can be programmatically changed. This read-only property is set when a media item is loaded into a MediaElement. If the referenced media item is being streamed, this property will be set to false. If the referenced media item is being downloaded progressively, the CanSeek property will be set to true.

When the CanSeek property is set to true, you can set the Position property to any TimeSpan value. It's recommended that you use a TimeSpan within the NaturalDuration of a MediaElement. If you use a TimeSpan beyond the NaturalDuration, the MediaElement will jump to the end of the media item.

The Position is an important part of any media item—and so are the other common properties shared across audio and video files. These properties include NaturalDuration, CurrentState, CanPause, and AutoPlay. Additional properties are specific to the audio part of a media stream.

Occasionally, audio or video files will contain more than one audio track. As an example, a song may have one track for the guitar, one for the drums, and one for the vocals. Usually, you won't work with these kinds of audio files. Instead, you may come across multilingual videos where each language has its own track. In both these situations, you can access the track-related information through the AudioStreamCount and AudioStreamIndex properties.

The AudioStreamCount and AudioStreamIndex properties give you access to the individual audio tracks of a media file. The read-only AudioStreamCount property stores the number of tracks available. The AudioStreamIndex property specifies which of the available tracks to play (or is playing). Neither of these properties means anything until the MediaOpened event has fired.

When the MediaOpened event is fired, the AudioStreamCount and AudioStreamIndex properties get set on the client's machine. When this occurs, the audio tracks in the media file are read. While these tracks are being read, a collection is being created in the background. When this collection is fully created, the AudioStreamCount property is set to match the number of tracks in the collection. Then, the AudioStreamIndex property is set to begin using the first track in the collection.

Alternatively, if the AudioStreamIndex property is set at design time, that track will be used. Either way, once an audio track is playing, it's important to make sure that the sound is balanced.

The Balance property enables you to effortlessly simulate sounds such as a wave gently lapping a sandy shoreline or a swirling wind. These types of sounds often involve sound shifting from one ear to the other; it would be startling if the sounds spastically jumped from one ear to the other. The balance of the volume across your ears makes these sounds much more natural.

With the Balance property, you can gracefully spread out your sounds by specifying a double-precision value between –1.0 and 1.0. If you set the property value to -1, you can project sound entirely from the left-side speakers. If you set the value to 1, the sound will leap from the right speakers. If you're seeking a balance between the left and right speakers, you set the value to 0.

This property is more than an enumerator between the left, right, and center positions. It gives you the flexibility to do things like project 70 percent of a sound from the right speaker by using a value of 0.7. The remaining 30 percent projects from the left speaker. As you can imagine, you can easily depict a lifelike audible environment. Sometimes it's nice to shut out the sounds of life—enter IsMuted.

Anything with an audio source should expose the ability to temporarily mute the audio. Thankfully, the MediaElement exposes an IsMuted property.

This property allows you to programmatically determine whether the sound associated with a MediaElement is audible. If a MediaElement is playing and this boolean property is set to true, the MediaElement will continue to play, but it won't be audible.

As a bool, the IsMuted property is all or nothing. Usually, you'll need to find a happy medium between audible and inaudible. Silverlight also gives you this type of control through the Volume property.

The Volume property is a double-precision, floating-point value that specifies the audible level of a MediaElement. This property value can range from an inaudible (0.0) all the way up to a room-shaking 1.0. The room-shaking capabilities are ultimately restrained by the user's computer volume. By default, the Volume value is in the middle of this range at 0.5.

The Volume property is one of the five properties that address audio-related features. The other properties are the IsMuted, Balance, AudioStreamCount, and AudioStreamIndex properties. The MediaElement also exposes a pair of properties that are specific to the visual part of a media file.

Client-side playlists are defined as Advanced Stream Redirector (ASX) files—this is just a fancy name for a specific XML format. Because this format is XML, you can create a client-side playlist with your favorite text editor, Windows Media Player, or server-side application. Regardless of your choice, this file will always follow a common structure, which is shown in this example:

<ASX Version="3.0">
  <Title>Silverlight in Action Videos</Title>
  <Entry>
    <Title>Greetings</Title>
    <Author>Chad Campbell</Author>
    <Ref Href="http://www.silverlightinaction.com/video1.wmv" />
  </Entry>
  <Entry>
    <Title>City Scape</Title>
    <Author>Dan Herrenbruck</Author>
    <Ref Href="http://www.silverlightinaction.com/video2.wmv" />
  </Entry>
</ASX>

This example shows a pretty basic client-side playlist that uses a small portion of the full ASX schema. This segment isn't that far off from the full schema supported within Silverlight. Silverlight only supports a subset of the full ASX schema, but this subset still provides plenty of elements that can be used to deliver a rich client-side playlist (see table 20.5).

This table shows the ASX elements supported within Silverlight. As you can see, an ASX file is more than a list of URLs that point to media files. The ASX file format gives you the opportunity to provide a lot of valuable metadata with a playlist. In fact, the ASX format lets you specify metadata for the media items within the playlist, so it's important to understand how to access that metadata.

The metadata for a media item can be found within a read-only property called Attributes. This member of the MediaElement class exposes the metadata as a Dictionary<string, string>. There are two interesting characteristics about this property that deserve mentioning.

The first is in regard to what metadata is exposed. Surprisingly, the metadata embedded within a media item isn't included. Unfortunately, there isn't an elegant way to get this information. The descriptive information stored within the ASX file is included, so if you're using client-side playlists you should provide as much metadata as you can.

The other interesting item is related to the lifecycle of the Attributes property. This property stores the metadata associated with an individual media item, so the Attributes property is cleared and repopulated each time a different track in an ASX file is started. If you're changing your UI based on the values within the Attributes property, you may consider doing this in the MediaOpened event. Alternatively, you may decide to bypass client-side playlists altogether and use a server-side playlist.

Server-side playlists are defined as .wsx files. These files are XML files that follow a specific XML format, which is demonstrated in the following sample .wsx file:

<?wsx version="1.0"?>
<smil>
  <seq id="sq1">
    <media id="advertisement1" src="advertisement1.wmv" />
    <media id="movie" src="myMovie.wmv" />
    <media id="advertisement2" src="advertisement2.wmv" />
  <seq>
</smil>

This XML example shows a basic .wsx file. This playlist uses three of the elements supported by the SMIL format in Silverlight—Media, Seq, and Smil. Silverlight supports a total of five elements, which are listed and described table 20.6.

The elements listed in the table give a content administrator the flexibility to control how content is distributed. To distribute this content, you use a MediaElement to reference the .wsx file.

After your .wsx file has been created, you can publish it on your server. You must publish a server-side playlist before a Silverlight application can use it. Although publishing a server-side playlist is beyond the scope of this book, connecting to one isn't. You can do this from a MediaElement as shown in this example:

<MediaElement Source="mms://www.silverlightinaction.com:1234/myPlaylist" />

This line of markup shows how to reference a server-side playlist from a MediaElement. You may have noticed that the playlist doesn't include the .wsx file extension. This extension usually gets removed during the publishing process. A MediaElement must use the MMS protocol to request a server-side playlist. This playlist can be used to stream content but can't be used to serve downloadable content in Silverlight.

Server-side playlists provide a way for content administrators to control the distribution of their content. Client-side playlists turn that control over to the requesting application. Either way, both options give you a way to distribute that web-based mixtape you've always wanted to send. Of course, playlists (and media players in general) aren't very useful without providing control over the playback.

When creating raw video, the first step is to set up the video stream parameters. The parameters include things such as the height and width of the frame, the number of frames per second, and the actual video format.

Silverlight supports a number of different video formats, each identified by a FourCC code. FourCC is a standard four-character code that's used to uniquely identify video formats. In addition to all of the existing formats (for example, H264 for h.264 video), two new formats were added specifically for use raw media and the MediaStreamSource API. Those are listed in table 20.11.

In the example in this section, we'll use the RGBA format to push raw pixels without any special processing or encoding. It's the easiest format to use, requiring no algorithm other than providing a single pixel with a single color. Listing 20.6 shows the video setup code for our simple white noise generator.

Example 20.6. Setting up the video stream

Listing 20.6 shows two functions: OpenMediaAsync and PrepareVideo. They've been broken up that way because OpenMediaAsync will also need to support audio later in this section.

When the class is wired up to a MediaElement, Silverlight will first call the OpenMediaAsync function. In that function, you need to tell Silverlight what streams are available , a single video stream in this case. Then you need to set up attributes for the duration of the video, infinite in our case, and whether you allow seeking. You take that information and pass it into the ReportOpenMediaCompleted method to tell Silverlight you're ready.

The PrepareVideo method sets up some variables that will be used when we generate the samples. First, we identify the amount of time per frame. This can vary over the course of the video, but it'll be easier on the developer if you pick a constant frame rate. Then we set up a dictionary of attributes that identifies the format of the video and the dimensions of each frame . Finally, that's all packed into a MediaStreamDescription to be used when we start generating frames.

Once the video stream is set up, the next thing to do is to start pumping out frames to be displayed.

The main purpose of a MediaStreamSource implementation is to return samples. In the case of video, a sample is one complete frame, ready to be displayed. Listing 20.7 shows the GetVideoSample function, called by GetSampleAsync.

Example 20.7. Returning the video frame sample

The GetVideoSample function first checks to see whether we're approaching the end of the allocated video buffer. If so, it rewinds back to the beginning of the buffer. This is an important check to make, as you don't want to allocate a complete stream for every frame, but a stream can't be boundless in size.

Once that's done, I loop through the buffer, moving four bytes at a time (the size of a single pixel in the buffer) and generate a random pixel value. The pixel will either be almost white or almost black . When playing with the sample, I found that pure black and white was far too harsh, and these two slightly gray values looked more natural. Though not obvious here, when setting the pixel values you need to do so in Blue, Green, Red, Alpha (BGRA) order.

The next step is to write the buffer to the stream . In this simple example, I could've written the bytes directly to the stream and eliminated the buffer. But in anything more complex than this, you're likely to have at least two buffers (a read-from and a write-to buffer), and even more likely to have a queue of frame buffers used for preloading the individual frames.

Once the stream is populated, I then create the media stream sample , increment our time counters, and call ReportGetSampleCompleted to return the sample to Silverlight.

One interesting note in this is how sample time is used rather than frame numbers. The use of a time for each frame allows Silverlight to drop frames when it starts to lag behind. This was a key reason why I chose MediaStreamSource over other approaches in the Silverlight C64 emulator. When the user's machine is busy, or in case it's too slow to run the emulator at full frame rate, I continue to chug along and let Silverlight skip frames it doesn't have time to show. This helps keep everything in sync time-wise, which is crucial when you're also creating audio.

The first step in setting up the sound source is to modify the OpenMediaAsync function. That function currently includes a call to PrepareVideo followed by adding the video stream description to the list of available streams. Modify that code so that it also includes the audio description information as shown here:

...
PrepareVideo();
PrepareAudio();

availableStreams.Add(_videoDesc);
availableStreams.Add(_audioDesc);
...

Once those changes are in place, we'll add the PrepareAudio function to the class. The PrepareAudio function is the logical equivalent to the PrepareVideo function; it sets up the format information for Silverlight to use when reading our samples. Listing 20.9 shows the code for that function and its required class member variables and constants.

Example 20.9. The PrepareAudio function

The most important parts of this listing are the constants controlling the sample format . For this example, we're generating 16-bit samples, in two channels (stereo sound), at a sample rate of 44,100 samples per second: CD-quality audio.

Once those constants are established, they're used to figure out almost everything else, including the number of bytes per second and the block alignment . Once the WaveFormatEx structure is filled out with this information, I set it as the Codec Private Data using its little-endian hex string format. Finally, I create the audio description from that data, to be used when reporting samples back to Silverlight.

The final step is to output the audio samples. This requires generating the individual samples and returning them in chunks of predefined size. We'll use a random number generator to generate the noise, much like we did with video. Listing 20.10 shows how to fill a buffer with audio and return those samples to Silverlight.

Example 20.10. Outputting audio samples

The process for generating the white noise audio sample is similar to generating the frames of video. But instead of having a fixed-width x height buffer we must fill, we can generate as long or as short a sample as we want. This is controlled by the audio buffer size set in code. In general, you want this number to be as low as possible, as larger numbers typically introduce latency as well as skipped video frames—the system is too busy generating audio to show the video frame. But set the number too low, and the audio will stutter. If you find the white noise stuttering on your machine, up the buffer to 512 or so and see how that works for you.

Once the buffer size is established, I perform the same stream overrun check that we did for video, and for the same reasons. Then, I loop through the buffer, 2 bytes (16 bits) at a time, and generate a white noise sample. Once the sample is generated, I get the 2 bytes from it using a little bit-masking , and then write those bytes into the buffer. Once the buffer is filled, it's copied into the stream and the sample response built . After incrementing the time counters, the last step is to report the sample to Silverlight .

If you run the application at this point, you should have a short delay while the startup code is executed and the Silverlight internal buffers are filled, followed by simultaneous audio and video white noise. On the surface, this may not seem impressive. But when you consider that the video and audio is completely computer generated, it's considerably more impressive.

Raw audio and video also allow you to display any type of media for which you can write a decoder. Much of the IIS Smooth Streaming client for Silverlight, for example, is written using a custom MediaStreamSource implementation. Though writing a typically hardware-implemented 1080p HD codec in managed code may not lead to good performance, there are many other popular formats which don't have native Silverlight support, but which would benefit from a custom MediaStream-Source implementation.

So far, we've seen a number of ways to get video and audio into Silverlight. The easiest, of course, is to use a video format Silverlight supports and just point the MediaElement to it. Another way is to use the MediaStreamSource class to implement your own managed codec. One final way to get video and audio into Silverlight is to use the webcam and microphone APIs. A segment of the API, especially the VideoSink and AudioSink classes, is conceptually similar to the MediaStreamSource code we've completed in this section, but thankfully much simpler.

Silverlight allows the user to set the default webcam and default microphone. This is done by right-clicking on any Silverlight application and selecting the Silverlight menu option. Alternatively, the user can open Microsoft Silverlight from his program shortcuts. Once there, select the Webcam/Mic tab and pick from the list of available options. You'll see a preview of the webcam to the left and an audio level meter for the microphone on the right. Figure 20.10 shows the configuration dialog.

The settings start out using default capture devices on your machine. You can change it from there. The changes will globally affect all Silverlight applications that use the webcam or microphone.

With the default device set in Silverlight, it's time to write a little code to capture information from the default webcam.

Figure 20.10. Silverlight default webcam and microphone tab in the Silverlight settings dialog. Either that image is horizontally stretched in an unflattering way, or I need to lay off the chips.

Webcams typically support a number of resolutions and video formats. I have a Microsoft LifeCam Cinema on my PC, and it handles everything from the smallest of postage stamps to 720p HD video. As the capabilities vary from model to model, you'll need a way to query the webcam to identify its supported video formats.

The VideoCaptureDevice class contains a number of properties. The one of interest to us in this case is the SupportedFormats collection. SupportedFormats is a collection of VideoFormat objects, the properties of which are displayed in table 20.12.

To query the formats for my own camera, I injected this bit of code into the listing at the beginning of this section:

foreach (VideoFormat format in camera.SupportedFormats)
    Debug.WriteLine(
        format.PixelWidth + "x" +

        format.PixelHeight + " at " +
        format.FramesPerSecond + " fps " +
        format.PixelFormat.ToString());

The resulting list included (among many others) these entries:

640x480 at 30.00003 fps Unknown
160x120 at 30.00003 fps Unknown
160x120 at 30.00003 fps Unknown
1280x720 at 15.00002 fps Unknown
1280x720 at 15.00002 fps Unknown
960x544 at 30.00003 fps Unknown
960x544 at 30.00003 fps Unknown
800x448 at 30.00003 fps Unknown
800x448 at 30.00003 fps Unknown
800x600 at 30.00003 fps Unknown
...

Oddly enough, the pixel format came across as Unknown in all cases. Try it with your own webcam and the results will likely vary. Once you see a video format that works for you, you can choose it by assigning it to the DesiredFormat property of the VideoCaptureDevice. This example uses a LINQ expression to grab the first format with the highest resolution:

var format = (from VideoFormat f in camera.SupportedFormats
              orderby f.PixelWidth * f.PixelHeight descending
              select f).FirstOrDefault<VideoFormat>();
if (format != null)
  camera.DesiredFormat = format;

That will pick the format with the highest total pixel count. You can modify the statement to pick just the largest width, or the largest size that will fit within a given box, and so forth. Figure 20.11 shows the 720p HD version of the webcam shot from the previous listing.

Figure 20.11. Webcam screen shot at 720p HD, selected using the Desired-Format property and LINQ. I'm practicing my raised-eyebrow news anchor face. I'll try harder next time. Dig the C128 in the background!

One reason you may want to capture at a high resolution is to support the capturing of still images. The Silverlight webcam API allows you to use the webcam as a simple still image camera, returning individual images as WriteableBitmap instances.