CHAPTER 16
Windows Media

This chapter is about Windows Media the format, not about media on Windows the platform. Windows playback of other media types is covered in Chapter 28.

Windows Media is Microsoft’s first-party digital media platform. Like many a Microsoft technology, the first few versions (known as NetShow) were weak. But over time, it matured into a true powerhouse and the eventual victor of the turn-of-the-millennium QuickTime—RealMedia—Windows Media streaming wars. And needless to say, it’s something I’ve worked with quite a bit myself, both before and during my own time at Microsoft. But I was a fan well before I worked on it, and I think it remains the most complete end-to-end media delivery architecture. Windows Media does a great job of handling live broadcast and on-demand video, as progressive download and streaming files on PC (and not just Windows) and Macintosh computers as well as mobile devices. Windows Media was the first architecture with integrated digital rights management (DRM), and remains the most broadly used and deployed DRM platform.

Microsoft’s strategy is much less Windows Media—centric these days. Windows Media certainly is still supported and enhanced, but MPEG-4 and H.264 are also supported in Silverlight, Windows 7, Xbox, Zune, and beyond. The Windows Media Video 9 codec has been standardized as VC-1 and is used outside of Windows Media, including in Smooth Streaming .ismv files and on Blu-ray optical discs.

Windows Media is most dominant in the enterprise market, since it is preinstalled on the Windows desktops prevalent in corporations, with streaming supported from their Windows Server 2003 and 2008 installations. Windows Media is also very broadly used for downloadable and streaming video, particularly of premium content in Europe and Asia. In the United States, Netflix, Blockbuster, CinemaNow, and the Xbox and Zune marketplaces all use Windows Media.

The biggest historic weakness of Windows Media has been relatively weak cross-platform and cross-browser support; if your content ran on a Microsoft operating system and Internet Explorer, everything was generally great, but otherwise you could run into many limitations. This was fine in corporate environments, but was a problem when targeting a broad consumer audience. Addressing that limitation was a big reason for the development of Silverlight, and Silverlight’s cross-platform and cross-browser compatibility has been central to its own success. Microsoft also makes underlying technologies for Windows available for license as porting kits including source code for both encoding and playback. Third-party products like Flip4Mac and Episode incorporate these.

While Windows has had a Media Player for nearly two decades, the current era of Windows Media really started in 2003 with the release of Windows Media 9 Series, which included dramatically improved versions of the codecs, player, server, and file format. The player, server, and codecs have been updated multiple times since then, but still maintain broad compatibility with the 9 series launch. For typical web streaming use, it’s easy to maintain perfect compatibility with WMP 9 and later while taking advantage of improved authoring tools and servers.

Why Windows Media

There are a variety of reasons why Windows media is so broadly used, as discussed in the following sections.

Windows Playback

If you’ve got content targeting the Windows platform (which is about 90 percent of all PCs in the world) Windows Media is universally supported with a high level of functionality. As long as users are on at least XP Service Pack 2, they’ll have Windows Media Player 9 at a minimum, and so will be able to handle the 9 Series technologies.

Enterprise Video

Windows Media, like Windows itself, has placed a large emphasis on the enterprise, and provides unique functionality for large organizations. Being preinstalled on most corporate and government desktops makes deployment easy, and it supports multicast, critical for being able to deliver live corporate events across a WAN.

Interoperable DRM

Windows Media DRM is the most broadly deployed content protection technology, and the only one so far approved by all the major film studios. Beyond WMP, it’s also supported by a wide variety of devices, and on Mac via Silverlight in many markets, Windows Media is required for Hollywood-licensed content for DRM.

Why Not Windows Media

Not Supported on Target Platform

The biggest reason not to use Windows Media is if your users don’t have support on their target platform. Out of the box, Macs and Linux PCs don’t, nor do many CE and portable media devices. For web video, Silverlight is changing this dynamic quickly by supporting WMV cross-platform, cross-browser. But as of this writing, Silverlight was still catching up with Flash’s ubiquity.

The Advanced System Format

The file format of Windows Media is called Advanced Systems Format, or ASF, and it’s quite a bit more complex than the older AVI format. The principal design goal of ASF was for it to be highly efficient for a “bit-pump” server, minimizing the file parsing required and thus CPU requirement per stream. This has been a major part of Windows Media strength in server scalability, allowing a single box to deliver more content simultaneously than for competing formats.

ASF mainly contains video and audio, plus file- and time-based metadata. ASF files can also include markers and scripts mainly used for closed captions and to trigger web browser or Silverlight events. Some enterprising vendors have successfully implemented other data types in ASF; Accordent embeds JPEG slides inside ASF streams with their presentation products.

ASF can also contain multiple video and audio tracks. This enables bitrate switching (called “Intelligent Streaming”) and multiple audio languages.

An ASF file requires indexing to enable random access. In most cases, indices are dynamically generated by the Windows Media Format SDK as the file is created, but if a file doesn’t have an index (for example, a live broadcast currently in progress), it will need to be indexed later (the free Windows Media File Editor can do this).

The original file extension for ASF files was, unsurprisingly, .asf. However, because Windows uses file extensions to discriminate among file types, there was no way to assign different icons or default actions to video files (which couldn’t play on music devices back then) and audio files (which could). So, with the introduction of Windows Media v7, .wmv (Windows Media Video) and .wma (Windows Media Audio) extensions were created. These don’t represent any change to the underlying file type, just to the extension, and so you can generally mix and match extensions with no fundamental problem. Windows Media can also use metafiles that reside on web servers for server-side playlists and redirection. These metafiles link to the actual media. The original extension for those metafiles was .asx, but the more recent (although less often used) .wax (audio) and .wvx (video) discriminate between audio and video metafiles.

Windows Media Player

The canonical player for Windows Media content is good old Windows Media Player (WMP). It has a long heritage going back to 1991 and the Media Player application included with the Multimedia Extensions to Windows 3.0. There’ve been plenty of versions since then, with the oldest version still seen widely being WMP 9, preinstalled with Windows XP Service Pack 2, and an optional download back to Windows 98 and 2000. WMP 9 was part of the huge “Windows Media 9 Series” wave of products, and remains a fine baseline to target for broad compatibility.

The current versions are WMP 11, preinstalled with Vista and a recommended download for XP, and WMP 12 in Windows 7. (WMP 12 isn’t available for older versions of Windows), and contains much broader format support, including MPEG-4 and H.264.

Windows Media Player support of formats beyond Windows Media is covered in Chapter 28.

Windows Media Video Codecs

Since the Windows Media Video 9 codec was standardized as VC-1, detailed information on using it is in Chapter 17. This section will discuss the particulars of VC-1 in Windows Media.

Windows Media Video 9 (“WMV3”)

Windows Media Video 9 (WMV 9) has been the mainstream choice for most Windows Media content since the 9 Series launch. There have been five generations of backward-compatible encoder enhancements that provide quality and encoding speed improvements while maintaining full compatibility.

In general, the only time you’d use a codec other than WMV 9 for web video use is if you were trying to stream native interlaced content or have a single bitstream compatible with Windows Media Player and Smooth Streaming (and hence would use WMV 9 Advanced Profile, described shortly) or screen capture content (potentially using WMV 9.2 Screen, also described in this chapter).

Profiles

For computer playback, WMV 9 Main Profile is the broadly compatible high quality choice. Simple Profile is a simplified version targeting low-power devices like mobile phones. This simplicity means it requires less horsepower to play back, but it also requires more bits to provide equivalent quality to Main Profile since Main has some additional tools to improve compression efficiency (particularly B-frames and the in-loop deblocking filter). That said, most devices from Windows Mobile 2003 forward support Main Profile, although potentially requiring a lower bitrate. And mobile devices with hardware decoding like more advanced phones and the Zune handle Main Profile with aplomb. See Figure 16.1.

Figure 16.1 The Simple/Main profile switch in Windows Media Encoder.

Encoding tools that don’t provide a profile control are Main Profile only.

Windows Media Video 9 Advanced Profile (“WVC1”)

Windows Media Video 9 Advanced Profile is a non-backward-compatible enhancement to WMV 9 focused on professional video applications. The two most important additions are native interlaced support and transport independence enabling use in formats other than ASF. For traditional WMP playback, neither is needed. WMP 11 on both XP and Vista don’t automatically convert interlaced WMV9 AP to progressive on playback unless a registry key is set, so interlaced playback is really only of use in customized apps and in dedicated players (software Blu-ray players being the most common). The only addition that significantly helps quality is the ability to use Differential Quantization on I-frames, discussed in the next chapter.

Table 16.1 Windows Media Video and VC-1 Equivalents.

Windows Media Codec Name	WMV 9 Codec Profile	VC-1 Profile Name
Windows Media Video 9	Simple Profile	VC-1 Simple Profile
Windows Media Video 9	Main Profile	VC-1 Main Profile
Windows Media Video 9	Complex Profile (deprecated)	n/a
Windows Media Video 9	Always Advanced Profile	VC-1 Advanced Profile
Advanced Profile

Windows Media Video 9 Screen

The WMV 9 Screen codec was designed for efficient compression of screen recordings. It’s a special-use codec—very efficient for this task, but not for normal video. Screen is most efficient with simpler, flat graphics like Windows 2000 or the “Windows Classic” themes in XP, Vista, and Windows 7. However, richer graphical environments like Vista and Win 7’s Aero Glass include lots of gradients and transparencies, and normally encode more efficiently using standard WMV 9.

For efficiency, WMV Screen needs access the uncompressed RGB source video of the screen shots—it doesn’t work well when compressing from screen shots that have had any lossy encoding already applied to them, or even simple resizing. It’s typically used in conjunction with lossless screen recording products like TechSmith’s Camtasia. On a fast computer, it’s possible to use Windows Media Encoder to record or even broadcast live screen activity with the screen codec, but not recommended. As discussed in Chapter 5, newer DWM-based screen capture technologies like that in Expression Encoder 3 provide much better performance. Compared to other screen capture codecs, WMV 9 Screen is unique among screen codecs providing full support for 2-pass VBR and CBR encoding, making it possible to use for streaming (live and on-demand). Still, for most modern screen recordings, VC-1 is a better choice. Expression Encoder’s screen recording targets VC-1 only.

The Screen codec is Windows-only, and not supported in Silverlight, Flip4Mac, and other porting kit–based players.

Windows Media Video 9.1 Image

Also known as PhotoStory, the WMV 9.1 Image codec isn’t a video codec per se. Instead, it stores still images and metadata describing how they can be animated for a slideshow, including transitions. For this special class of content, it can be much more efficient than a typical video codec. It’s not widely supported outside of Windows; Silverlight and Flip4Mac don’t play it back.

Legacy Windows Media Video Codecs

Given Windows Media’s longevity, it’s had many different codecs over the years. While WMV 9 is all that need be authored these days, I’ll list the details of the legacy codecs in case you stumble across them. In general, Microsoft-derived WMV 9 decoders like those in Silverlight and Flip4Mac can also decode WMV 8 and WMV 7, but not earlier codecs.

Windows Media Video 8 (“WMV2”)

Windows Media Video v8 was the predecessor to WMV 9, naturally. It was about 15–20 percent less efficient than the original version of WMV 9, and is much further behind current implementations.

It was notable for having a very fast encoder compared to the original releases of WMV 9, but on modern hardware, that’s very rarely a concern anymore.

Windows Media Video 7 (“WMV1”)

Like WMV 8, WMV 7 was yet faster to encode and yet less efficient to compress than its successor. Again, there no reason to make these files anymore. And this is where the 4CC version number offset started; they could have saved some confusion if it had been “WMV7”

MS MPEG-4 v3

This was the last Microsoft codec based on MPEG-4 Part 2 Simple Profile. The codec had three versions (v1, v2, and v3). The latter was on the market the longest, when it became clear that MPEG-4 part 2 was hitting a wall for how far it could be optimized, and thus the superior WMV codec series was begun. MS MPEG-4 v3 was the subject of the original “DivX ;-)” hack to enable its use in AVI files.

The encoder hasn’t been included in Windows since XP, although some video tools may install it as part of an older Format SDK.

MS MPEG-4 v1 and v2

These older codecs were the initial versions of Microsoft’s MPEG-4–derived codecs, and predate the final MPEG-4 standard. They offered substantially lower quality than MS MPEG-4 v3. Encoders for these haven’t been in the Windows Media SDK since 1999, so only very old tools can encode to them. MS MPEG-4 v2 was the last version of the codec that also functioned as an AVI codec, although the DivX hack makes MS MPEG-4 v3 work in AVI files.

ISO MPEG-4

MS MPEG-4v3 implemented a draft of MPEG-4 part 2, and wasn’t quite compatible with the final spec, so this ISO-compatible version was introduced alongside the initial WMV 7 codec. It didn’t see significant use, as Windows Media didn’t otherwise support the file format, stream, or audio codecs from MPEG-4, and WMV 7 outperformed it handily.

Windows Media Screen 7

Windows Media Screen 7 was an earlier, weaker version of Screen 9. It’s no more broadly supported, and thus shouldn’t be used anymore.

Windows Media Audio Codecs

The Windows Media Audio codecs are covered in detail in the next chapter.

Encoding Options in Windows Media

Data Rate Modes

Windows Media supports the full range of rate control modes discussed back in Video Codecs. Unusually, Windows Media supports 2-pass modes, including 2-pass VBR, for audio as well as video.

Constant Bitrate (CBR)

Windows Media tools normally define CBR buffers by duration of the buffer, but some VC-1 implementations use VBV in bytes.

Normally CBR is used for streaming, although Windows Media Services can stream VBR content as well, treating it as a CBR file at the peak bitrate. CBR is also required for live encoding for streaming.

1-pass

1-pass CBR is required for live streaming, of course. It’s also the fastest bitrate-limited mode available in Windows Media (the bitrate-limited VBR modes are 2-pass only).

Note that 1-pass CBR encoding can be significantly improved by the use of the Lookahead and Lookahead Rate Control features. In current Windows Media implementations, 1-pass CBR with those modes normally outperforms 2-pass CBR in quality as well as speed.

2-pass

2-pass encoding lets the encoder see into the future, ramping the bitrate up and down as needed to account for future changes. In theory, it should be better than 1-pass CBR, but 1-pass CBR recieved tremendous tuning for live streaming and now generally surpasses 2-pass CBR. The only reason you would get much use out of 2-pass CBR is with a very long buffer (more than eight seconds) or when only stock settings are used. This may change with future implementations.

The first pass runs much faster than the second pass or a single pass would be, so going to 2-pass doesn’t double encode time—it’s more typically a 20 percent increase.

Variable Bitrate (VBR)

The essential difference between CBR and VBR is that VBR files have a peak bitrate higher than the average bitrate. This lets VBR file be more efficient for a given file size, since it can redistribute bits throughout the file to provide optimal quality.

You can also think of the difference as “CBR maintains bitrate by varying quality, and VBR as maintaining quality by varying bitrate.”

1-pass Quality-limited VBR

1-pass (quality-limited) VBR uses a fixed quantization for I- and P-frames (with B-frames varying some by amount of motion). Each frame using as many bits as needed. File size varies tremendously depending on content, and there’s no limit on peak bitrate at all. Obviously, this makes quality-limited VBR a poor choice for content distribution. It’s mainly used for mezzanine files, and since it is 1-pass, it can be captured in real time on a sufficiently powerful box.

2-pass Bitrate VBR (Constrained)

Bitrate VBR (Constrained) is the optimal encoding mode for progressive download and file based playback where the peak bitrate can be higher than the average bitrate. Almost every web video (WMV) file not streaming from Windows Media Services should be encoded using Constrained VBR.

With older versions of the codec, 1-pass CBR was recommended for HD content due to a problem with occasional dropped frames with 2-pass at high bitrates. This was fixed in the v11 codecs, and 2-pass VBR can be safely used for HD content. This can result in big savings in file size at a given quality level.

As a rule of thumb, VBR becomes significantly better than CBR when the peak is around 1.5x or more the average bitrate.

2-pass Bitrate VBR

The unconstrained 2-pass Bitrate VBR mode is the same as Constrained VBR without any peak constraint. This means that playability isn’t predictable; the constrained mode ought to be used instead. While many tools expose this mode, it’s never the best choice.

Where Windows Media Is Used

Windows Media for ROM Discs and Other Local Playback

Windows Media is widely used for the video portions of Windows-only ROM discs, although web delivery is rapidly shrinking that market.

One common use of WMV for local playback is embedded video in PowerPoint presentations. This was frustratingly not cross-platform for over a decade, but current versions of Office for Mac 2008 are now able to play an embedded WMV using Flip4Mac without any trouble.

Windows Media for Progressive Download

Although Windows Media was originally targeted at streaming, it has become a great technology for progressive delivery.

There aren’t any particular tricks to encode for local playback of WM files, except to use a single video track instead of the Intelligent Streaming multiple bitrate mode, and to use 2-pass VBR Constrained for video and audio.

Windows Media for Streaming

Windows Media adopted the RTSP protocol for streaming many years back, dropping the older, proprietary Microsoft Media Server (MMS) protocol. Still, a URL for a streaming ASF file typically still starts with mms://. This serves a hint to Windows Media–incompatible players that they probably can’t play back the content, and to Silverlight and WMP to stream the file instead of trying progressive download. Windows Media Services (WMS) was built into Windows Server 2003 and is a free download for Windows Server 2008. RealNetworks’ Helix Server includes some Windows Media support, but it hasn’t been actively developed for some years and can have trouble with newer content and players.

Windows Media Services is a full-featured streaming server, familiar to administrators of Microsoft’s Internet Information Services (IIS). It offers all the appropriate features of a modern server architecture. By default, it will fall back to TCP delivery if UDP is blocked by a firewall, and it can be configured to fall back to HTTP. The Intelligent Streaming MBR mode will correctly switch data rates even when being used in this mode.

On-demand streaming

The ideal encoding mode for on-demand streaming is CBR. The larger the buffer, the higher the average quality, but the greater the latency for both startup time and random access. If the server is using the Advanced Fast Start and Fast Cache features, random access will be reduced as the user’s connection speed goes up, so you can get away with bigger buffers.

Live streaming

Windows Media provides a good, mature system for doing live streams. Most folks have used Windows Media Encoder for capture, but other tools are much better today. Expression Encoder 3 offers a better experience at the WME price point. On the higher end, there are great real-time encoders from companies like Viewcast, Digital Rapids, Inlet, Envivio, and Telestream.

WMV can use up to eight processors for video compression on Windows 7 and Expression Encoder 3 (4 in older versions of Windows without EEv3), so if you’re doing live broadcasting, a hefty multiprocessor machine can substantially increase the maximum resolution, frame rate, and data rate you can target. 720p30 is realistic on a fast 4-core, and 720p60 and 1080p30 are possible with an Intel i7 8-core.

WME can also simultaneously capture and serve video from the same computer. However, unless you’re going to have more than a handful of viewers, it’s best to send a single copy of the stream to a dedicated server, so CPU cycles don’t get gobbled up by serving instead of being used for encoding.

Intelligent streaming

Intelligent Streaming is Windows Media’s Multiple Bitrate (MBR) technology. It bundles different video and audio bitstreams of the content into a single WMV file, with the server delivering just the video and audio the player requests based on available bandwidth. Different audio streams can also be used for different languages

The streams can vary in bitrate, frame size, frame rate, and buffer, but must share the same codec.

Intelligent Streaming was fine on the server side, but like all classic streaming MBR technologies, it was hard for the player to know which bitrate to use. While it was easy to figure out when the requested bitrate is too high (buffering), there wasn’t any good way to figure out if there was extra bandwidth to increase bitrate. Also, any bitrate switch would yield several seconds of buffering, which isn’t ideal.

Silverlight’s Smooth Streaming is a much more robust solution to the problems that Intelligent Streaming targeted, and is discussed in Chapter 27.

Windows Media for Portable Devices

Windows Media has long been supported on Microsoft’s portable devices like Windows Mobile phones and Zunes, plus devices from Nokia and other vendors.

Of course, many mobile devices are limited in processing power, memory, storage, and bandwidth compared to desktop computers.

All Windows Mobile devices since 2003 have supported WMV 9 Simple and Main Profile plus WMA Standard, Voice, Pro, and Lossless. However Advanced Profile and WMA 10 Pro’s LBR modes are not as widely supported.

For a device that supports WMV and H.264, a good WMV 9 Main Profile encode can sometimes retain more detail at moderate bitrates than H.264 Baseline Profile,

Embedding Windows Media in a Web Page

Windows Media Player is available as an OCX (Object Linking and Embedding [OLE] Control EXtension component; glad you asked?) for embedding in Internet Explorer, the primary method Windows Media files have been embedded in web pages. The WMP OCX works well on Windows, offering full hardware acceleration for playback, DRM, etc. It’s commonly used that way inside IE-only corporate environments. But that hasn’t addressed consumers with Macs, Linux, or other browsers.

For Firefox users on Windows, Microsoft provides a plug-in that embeds the WMP OCX in a Firefox-style plug-in. Its permanent home is: http://port25.technet.com/pages/windows-media-player-firefox-plugin-download.aspx

Mac users can use the free version of Telestream’s Flip4Mac from Flip4Mac.com. It’s based on the Windows Media porting kit, and offers increasingly good compatibility with existing sites, although complex JavaScript can sometimes cause trouble.

And if course Silverlight provides a cross-platform, cross-browser mechanism to embed Windows Media content. However, Silverlight requires a Silverlight app to control the media playback; it won’t used on WMP OCX-HTML targeted web pages.

Windows Media and PlayReady DRM

Digital rights management is required by many content owners to protect against piracy. Windows Media DRM was released back in 1999, and ran through WMDRM 10. Its successor technology PlayReady, which supports other formats including MPEG-4 and H.264.

The basic model of WM DRM is similar to other DRM solutions, like Apple’s FairPlay. The content is encrypted, and can’t be decrypted without a license key. The license key itself can be delivered by a server at play time, securely stored on the computer, or even stored in the file itself.

Different players can support more or fewer rules on how the content is used. Silverlight 2 and 3 really only support “Play Now” and don’t offer a way to store the local key. But Windows Media Player offers the full functionality. Some of these rules include:

From a compression workflow perspective, DRM can either be applied as part of the encoding process (if the encoding tool integrates DRM) or an unencrypted file can then be encrypted later in the workflow.

And from a business perspective, Windows Media DRM is the only broadly supported DRM available for license and approved by the Hollywood movie studios. In markets not large enough for publishers to negotiate one-off deals with each studio, WMDRM is often the only option to publish premium content.

Windows Media Encoding Tools

Windows Media and the VC-1 codec are supported by a vary wide array of tools, but most of them incorporate one of a few underlying SDKs. That’ll determine the features you have available. And in some cases, the same tool may offer different quality and performance depending on the version of Windows or Windows Media Player is installed.

Products using the VC-1 Encoder SDK don’t have any dependence on the OS for video, but still do for audio.

VC-1 Encoder SDK

The VC-1 Encoder SDK is a professional-grade implementation of VC-1. It supports encoding for Blu-ray and Smooth Streaming as well as WMV. The SDK, its features, and the tools that use it are covered in detail in the following chapter. But the important takeaway is the VC-1 Encoder SDK encodes faster and with higher quality than the Format SDK, and doesn’t require registry keys for advanced parameters.

Windows Media Format SDK

Windows Media Encoder and other older WMV encoders are graphical front-end to Microsoft’s Windows Media Format SDK. The actual codec version you’re running is determined by the Windows Media .dll (Dynamic Link Library) files on your system (Figure 16.2). Most people get those updates bundled with Windows Media Player, so as long as you have the most recent WMP version for your platform (that’s WMP 11 for XP and Vista) and the encoding hotfixes listed next—or, better yet, Windows 7—you’re good to go.

Figure 16.2 WMV9 PowerToy’s codec DLL information.

For vendors of compression tools, it’s always good to include the redistributable installer with your product, to ensure that users have the current version.

I’ve see far too many production encoders are still running WMP 9 and thus the 9.0 SDK. That’s a codec implementation from 2003, and will offer much lower quality and performance than the current versions deliver.

Windows XP, Vista, or Server 2008: Format SDK 11

Windows XP shipped with Windows Media Player XP, a flavor of WMP 7. However, the almost universally installed Service Pack 2 included WMP 9. WMP 11 is also available for XP, and is the minimum version seen in the wild today.

Windows Vista shipped with WMP 11 preinstalled, so no update is required beyond hotfixes listed here.

On Windows Server 2008, Windows Media Player, and hence the Format SDK (FSDK), is installed with the Desktop Experience feature; that’s installed through the management interface like all other Server 2008 features. You’ll want to enable Desktop Experience on any encoder box (and in general for any 2K8 box you’ll be using as a workstation).

Windows Server 2003: Format SDK 9.5

The most recent version for Windows Server 2003 is the Format SDK 9.5 (still better than 9.0), released with Windows Media Player 10.

Running on Server 2003 means no access via the SDK to the SMPTE-compliant “WVC1” flavor of Windows Media Video 9 Advanced Profile, nor WMA 10 Pro LBR modes (the 32–96 Kbps range). The video codec is also only 2-way threaded instead of 4-way threaded, slower in general, and lacks the advanced registry keys.

If you need to encode on a Server OS, you should use a VC-1 Encoder SDK-based product or upgrade to Server 2008 (ideally both!). However, WMA 10 Pro will still be unavailable.

There have been reports of using the WMFDist11-WindowsXP-X86-ENU.exe installer from FSDK 11 set to XP compatibility mode to install the FSDK 11 .dll files onto Server 2003. While there aren’t known issues with this, this isn’t a supported configuration for Server 2003, and Microsoft wouldn’t support issues when running with mismatched .dll files.

Windows 7

Windows 7 comes with updated Windows Media .dlls, but not a new Format SDK; they’re only available in Windows 7 and the Windows 7 based Windows Server 2008 R2. It doesn’t have any new registry keys or configurations, but it has two big performance improvements:

Windows 7 also has a quality improvement for noisy textures, particularly in shadows, reducing cases where noisy textures wind up getting turned into motion vectors. This reduces the appearance of “swirling” noise.

The same improvements are included in Expression Encoder 3, and hopefully soon in VC-1 Encoder SDK. See Figure 16.3.

Figure 16.3 (A) A challenging frame from Windows Media in 2003 using the first-generation VC-1. (B) And in 2009 with the fifth-generation implementation.

Low-Latency Webcasting

While Windows Media offers 15–20-second end-to-end delay by default, it’s possible to drive it down to 2–3 seconds with best practices on a good network. The critical thing is to tune the encoder, server, and player latency together.

Latency is the delay between when video enters the encoder and is seen on the user’s display. It’s sometimes called “broadcast delay.” Latency is something that doesn’t matter at all in some markets, and matters a lot in others.

The primary reason we have latency is buffering for quality and reliability. By having the server wait several seconds after a video packet is received before sending it out, it can smooth out peaks and valleys in the data rate, and can detect and resend dropped packets before their display time. Buffering in the player averages out data rates and allows recovery of dropped packets as well.

Large buffers were extremely important in the modem era, and the defaults can deliver highquality content over a variety of networks. But when minimizing the latency is important and reliable networks are available, the delay can be reduced substantially.

Since total latency is the sum of encoder latency, server latency, and player latency, plus packet transmission time, improving latency requires tweaks to the encoder, server, and player in parallel.

Encoder Latency

WME enables video encoding latencies down to one second, and EEv3 can go down to 0.1 second. Using Lookahead or Lookahead Rate Control (LRC) will increase latency beyond the buffer value (by the GOP duration for LRC).

Windows Media includes the “Low Delay” audio codecs for both WMA and WMA Pro. This can be recognized by having bitrates 1 less than the typical (like 127 instead of 128); actual bitrates are the normal (so 127 really uses 128Kbps). The specific low-latency codec modes are listed for each codec in Chapter 18.

Figure 16.4 Expression Encoder’s audio codec list. The “–1” settings like 127 are Low Latency, and minimize encoder-side buffering.

Server Latency

WMS features like Advanced Fast Start can dramatically reduce latency for on-demand content, but don’t apply to live streaming. Server buffering can be turned off entirely in WMS for Windows Server 2003 and 2008 (Figure 16.5), yielding a significant drop in latency (and a drop in robustness to dropped packets).

Figure 16.5 The Windows Media Services dialog to turn off server-side buffering.

Player Latency

Normally the player settings aren’t under control of the encoder or server. By default, WMP picks a buffer size based on measurements of network and stream performance. However, it’s possible for the user to lower the buffer size in the player’s options. This can help reduce latency when watching streams with a good connection, but could produce pauses in the video when watching video from the general Internet.

Silverlight makes the player buffer size a controllable parameter, so an optimal setting for a given stream can be specified. This is controlled by the BufferingTime parameter in a Silverlight MediaElement. See Figure 16.6.

Figure 16.6 XAML code for turning off client-side buffering.

Encoders for Windows Media

There are lots of tools available for Windows Media. The technology has been available for years, and SDKs have been free of charge for Windows all along. Essentially every general-purpose compression tool on Windows supports Windows Media, as do many on other platforms.

I’m covering WMV-centric products in this chapter. More advanced encoders with deeper VC-1 support like Fathom, Rhozet, and CineVision PSE are covered in Chapter 17.

Expression Encoder

Expression Encoder is Microsoft’s current first party compression tool for Windows Media. With version 3, it’s largely replacing Windows Media Encoder. EE also supports Smooth Streaming and H.264 output, as described in the relevant chapters.

Expression Encoder 3 is now available in a free version that supports the WMV features of WME. There’s also a $49 upgrade with additional codecs and format support, including:

EEv3 includes all the Windows 7 and VC-1 Encoder SDK improvements, so its WMV encoding is top notch in quality and performance. Better yet, it provides the settings in a humane manner, offering drill-down to features as needed, but also offering a simpler gradient of Faster to Better presets.

It’s also the first lower-cost live encoder to include the full VC-1 Encoder SDK, including dynamic complexity and lookahead rate control (full details in the VC-1 chapter). But that basically means it’ll automatically tune to the best quality your hardware can deliver, and will offer a lot better quality WME at the same bitrate.

Expression encoder notes

Figure 16.7 Expression Encoder 3, in its glory.

All that said, Expression Encoder is really focused on being a high-touch replacement for WME. There are many industrial-grade WMV encoders that use the same underlying VC-1 Encoder SDKs for mission-critical live encoding or high-volume on-demand encoding.

Windows Media Encoder

Windows Media Encoder has long been the most used encoder in the industry. Part of this was due to its deep integration with the Format SDK and implementation of the broad sweep of Windows Media features. And, perhaps as importantly, it was free. WME also has an “Encoder SDK” that enables simpler automation of live and on-demand encoding than the more complex Format and VC-1 Encoder SDKs.

However, the era of Windows Media Encoder is coming to a close, for a few reasons:

Now, the lack of mainstream support on Windows 7 doesn’t mean it doesn’t work; it seems to work just fine. But Microsoft isn’t obligated to fix any nonsecurity bugs going forward. Companies with existing WME workflows can continue to use it, but really should be looking to migrate to EE or commercial encoder product.

WMCmd.vbs

The ungainly named WMCmd.vbs is a CScript script that automates the Windows Media Encoder SDK from the command line, and it’s installed by default in C:Program FilesWindows Media ComponentsEncoderWMCmd.vbs. However, if you plan to use it, download the enhanced version from http://www.alexzambelli.com/wmv that includes full programmatic support of the various registry key options. For example, this is my old script for turning a 16:9 .avs from a DVD into a high-quality file playable on a Zune 2 and pretty much anywhere else. Needless to say, I switched to using Expression Encoder as soon as it was available (where I could just pick the “Zune 2 A/V Dock” preset): cscript “C:Program FilesWindows Media ComponentsEncoderWMCmd.vbs” -input MyDVD.avs -output MyDVD_Zune2.wmv -a_codec WMASTD -a_mode 4 -a_setting 128_ 44_2 -a_peak 192000 -v_codec WMV9 -v_mode 4 -v_keydist 4 -v_bitrate 2000000 – v_peakbitrate 3000000 -v_peakbuffer 4000 -v_performance 80 -v_profile MP -pix-elratio 40 33 -v_msrange 0 -v_mslevel 2 -v_mmatch 0 -v_adz 2 -v_bframedist 1 -v_dquantoption 2 -v_mvcost 0 -v_loopfilter 1 -v_overlap 1

Flip4Mac

Flip4Mac WMV is a component for QuickTime on the Mac that adds WMV support to QuickTime. The free version of Flip4Mac is playback only, but the higher-end versions support import of WMV files to tools like Final Cut Pro, and export to WMV. The available versions are:

▪ Free:	Playback only
▪ Pro:	Import only
▪ Studio	Preset-based encoding only
▪ Studio Pro:	SD 1-pass only
▪ Studio Pro HD:	Full support for HD, 2-pass, and multichannel audio

Beyond import and export, Flip4Mac (seen in Figure 16.8) will emulate the WMP OCX embedding options so that those web pages work on the Mac. This support has evolved over the years and now supported many more web sites.

Figure 16.8 (A) Flip4Mac supports some advanced encoding modes, including interlaced VC-1. (B) Unlike Episode, Flip4Mac supports 2-pass audio.

Windows Media in Flip4Mac notes

Episode

Telestream’s Episode compression tools began life as Compression Master from PopWire. Originally a Mac-only product, they now have Mac and Windows versions. Episode for Windows is unique in that it uses Telestream’s Porting–Kit derived encoder instead of either the Format SDK or VC-1 Encoder SDK, so quality and performance may vary from that of other tools on the same machine. See Figure 16.9.

Figure 16.9 The Episode “WMV 9” dialog. Unlike FSDK-based tools, it exposes the important B-frame parameter.

Windows Media in Episode notes

A few other notes on Flip4Mac:

WMSnoop

WMSnoop’s not really an encoding utility, but it’s cool enough I wanted to mention it. It’s a free WMV analysis tool from Sliq Media Technologies that provides useful information about WMV files, including a cool data rate graph. See Figure 16.10.

Figure 16.10 WMSnoop, a handy freeware WMV analysis tool.