CHAPTER 20
RealMedia

RealNetworks (then Progressive Networks) founded the streaming industry as we know it. RealPlayer v1 was launched in 1995 and was the first real-time streaming media player. Video was introduced in RealPlayer 4 in 1997, and the company quickly dominated streaming video as it had done for streaming audio. While there were other formats for distributing audio and video on the Internet, RealNetworks was the innovator and the market share leader for streaming delivery until Windows Media displaced it in the early 2000s.

Historically, the main sources of Real’s revenue were server software and the Plus version of the player software. But competition from full-featured free players like WMP forced Real to put more functionality into the free version of the player, reducing the truly useful features in the upsell version. And both Windows Media Services and QuickTime Streaming Server were free, eroding the rather massive per-user pricing RealServer used to demand.

Thus, RealNetworks has long-since shifted to content as a revenue driver, particulary the Rhapsody music service, SuperPass subscription service, and RealArcade casual game service. In parallel, they offer their Helix multiformat severs and hosting services, although they open-sourced much of Helix in 2003. The Helix brand extends to open-source implementations of the RealMedia SDK and RealMedia Producer compression tool.

So, while RealNetworks remains a vital company, their proprietary media format has been quickly fading from the scene. It’s still seen in some enterprise environments and in European markets, but has only a sliver of its former dominance. The single largest place it’s still used appears to be in Chinese Internet cafés, where RMVB (RealMedia Variable Bitrate) has long been the de facto standard.

Why RealMedia?

Since it’s not really a mainstream format anymore, if you need to use RealMedia, you proabably already do, and know why—typically an environment where RealMedia is the best format deployed on all target devices. RealPlayer was the first mainstream media player widely support on UNIX operating systems like Linux and Solaris, and the open source Helix Player has meant that RealVideo support is more common on Linux and even more obscure platforms.

RealMedia Format

The RealMedia format, often called a RealVideo file even if it only contains audio, has the extension .rm. RealAudio, or .ra, an older version of the format, is no longer in use. VBRcoded RealMedia files use the extension .rmvb, although they’re not really a new file format. That’s just an indication that the files shouldn’t be streamed, as they aren’t CBR.

RealPlayer

The current version of RealPlayer as of this writing is RealPlayer SP beta (replacing RealPlayer 11; see Figure 20.1) for Mac, Windows, and Linux. RealPlayer has long been widely ported, and it is currently also available for Linux, Palm, Windows Mobile, and Symbian. The Mac/Windows versions have broad support for formats beyond RealMedia, tapping into QuickTime and DirectShow APIs to handle anything the default media players can handle. Performance for complex H.264 files seems notably poor in SP compared to WMP and QuickTime Player, however.

Figure 20.1 The new RealPlayer SP has a very clean interface; a big change from RealPlayers past.

While recent RealPlayer versions have been generally clean and smooth to use, there’s lingering animosity over past versions. Historically, the two biggest objections were to the hard upsell to the paid version (currently $39.99), and the cluttered interface. A common user complaint was that people couldn’t find the free player’s link on Real’s site, making it potentially problematic to send a RealMedia link or file to a less tech-savvy friend or relative. The current player is quite clean and the upsell quite appropriate now.

The paid RealPlayer SP Plus version has reasonable value for its price, mainly focused on DVD/CD burning and device transcoding and sync. There are some gimmicky features I’m surprised that they’re still hawking, like “video EQ” to change brightness and saturation during playback. If the video was shot and compressed correctly to begin with, those filters could only be used to make the playback worse.

RealPlayer Mobile

There’s also an older RealPlayer Mobile available for Nokia’s 9200 Series and 7650, and Windows Mobile devices. RealVideo 9 support is only in a “Preview Release” despite having shipped with RealPlayer in 2002.

Helix DNA Client

The Helix DNA Client isn’t a product, but a technology that RealNetworks licenses to other companies, particulary in the mobile space, to build players on top of. This is a commercial product for Real, and is updated regularly unlike RealPlayer Mobile.

RealVideo for Streaming

RealVideo and RealAudio were both designed for real-time streaming, and were the best solution for streaming to modem users. That modem user has largely been abandoned by web video, but the underlying technology is still very capable of low bitrate streaming over lossy and high-latency connections.

SureStream

RealVideo’s SureStream supports dynamic switching between multiple data rates for both audio and video, up to eight each. The video bands can vary by data rate and frame rate, but not frame size. The audio bands can vary completely. Each band is given a target connection type, and when the RealPlayer connects, it starts with the stream that matches the specified connection type in the user’s player. Once the stream starts, the server will dynamically adjust it up or down to the largest streams that fit within the target data rate. Although audio and video are defined in pairs, SureStream may end up playing the audio track from a higherquality band than the video, if it can fit within the target bandwidth.

RealMedia streaming uses an implementation of the RTSP protocol, and requires Helix Server (formerly RealServer).

The biggest drawback to the Helix Server historically was its price. It’s gotten a lot cheapter over the years; for quite a while server pricing was a major motivation for content companies to switch to QuickTime or Windows Media. Helix Server pricing is now “call to talk to a representative”—it’s not clear what deployments cost.

Beyond RealMedia, there’s also the Helix Mobile Server targeting 3GPP video. They’ve done some innovative work around bitrate switching, and Mobile appears to be the larger focus for development and business.

RealVideo for Progressive Download

Earlier versions of RealVideo weren’t well suited to progressive download. That changed with the introduction of .rmvb and the 2-pass VBR mode of RealVideo 8. However, current versions do a fine job with it. RealVideo files encoded for progressive download shouldn’t use SureStream, of course. A file played locally will only play the highest data rate audio and video stream, so including more than one just makes the file larger.

RMVB remains in popular use in some Asian countries, primarily for pirated or usergenerated content.

RealMedia Codecs

RealVideo v10

RealVideo 10 is the current name for the RealVideo encoder; the bitstream is the same as RealVideo 9. RV 9/10 is somewhat like H.264 with in-loop deblocking and B-frames, but with a different quantization method. Following RealVideo tradition, it does a very nice job of getting soft before it gets blocky. Compared to RV9, RV10 has better rate control and motion search, and has been particularly improved for maintaining quality in high-motion scenes.

The RV9/10 decoder has been included in RealPlayer since 2003, so it’s a safe default codec for all desktop players now, although some older mobile devices are still stuck on RealVideo 8.

One unique control in the RealVideo codecs is four levels of image quality versus frame rate tradeoff. In order of higher frame rate, these are: Smoothest Motion, Normal Quality, Image Quality, and Slide Show. For most content, I find Smoothest Motion offers the best experience, and always use it by default.

RealVideo 10 is pretty fast to encode and decode. All-around, it was a very promising lowbitrate codec, but it arrived just as RealMedia was seeing a huge market share decline after the release of Windows Media 9 Series. This resulted in the latest RealVideo 10 encoding implementation not being broadly supported in commercial compression tools.

RealVideo NGV

RealNetworks had been working on a new codec called NGV (for Next Generation Video) for a few years, possibly to be called RealVideo 11. The project lead has moved on to another position inside of Real and there haven’t been any details about it since early 2008.

RealAudio Codecs

RealPlayer was an audio-only solution until v4, and it has always had a strong focus on audio. Instead of offering a few codecs with different settings, Real has always presented audio codecs as a list of content type, data rate, and channel pairs.

Some are marked “High Frequency.” These options support higher frequency response, but may increase artifacts when handling complex audio content. You may need to experiment to determine which version is better suited for your particular content.

SureStream works for audio-only content, so a Helix Server can automatically provide the optimum data rate version to a given user.

There are five basic audio codec families still in use.

RealAudio 10

RealAudio 10 is Real’s name for AAC (mostly –LC), and thus generally the best audio codec in RealMedia. Its only drawback is that it requires RealPlayer 10 for playback, which could exclude some machines that have gone a long time without upgrades, or old mobile players. It’s probably safe to use for most projects today. There are four modes:

RealAudio Voice

This is a speech codec. It’s a fine example of the genre, and is unusual in that data rates go up to 64 Kbps. That data rate is overkill for actual voice content, but 32 Kbps is an excellent choice for audio books and the like.

Stereo Music: RealAudio 8

The Stereo Music–RA8 codecs were introduced in RealPlayer v8.5. They were a substantial enhancement in providing cutting-edge compression at low data rates. There are actually two different sets of algorithms under the RA8 name: a version of Sony’s ATRAC-3 codec from MiniDisc players (enhanced for RTSP streaming) is used for 96 Kbps and higher data rates.

This provides quality from decent-for-streaming at 96 Kbps to archival at 384 Kbps. Below 96 Kbps, a RealNetworks propritary bitstream is used. In most apps that use the RealSystem SDK, the in-house codecs are referred to as “RA8” and the ATRAC-3—derived codecs are designated “RA 8” (with a space between the “A” and the “8”). Overall, RA8 Stereo is somewhere around MP3 and AAC-LC efficiency, although its low-bitrate artifacts are arguably less annoying than those of MP3.

Although I usually recommend using mono over stereo for lower data rates, the superior quality of the RA8 Stereo codecs over the mono Music codecs means that RA8 should be used instead.

RealAudio Surround

RealAudio Surround is a tweaked version of the normal RealAudio 8 codec that maintains the high frequencies and phase relationships required to maintain Dolby ProLogic matrixed surround sound for playback through a ProLogic reliever. Note this isn’t a true 5.1 solution; it just preserves existing surround information encoded in two channels.

RealAudio Surround requires higher data rates and delivers lower compression efficiency than the normal RA8 codecs, and should therefore be used only with ProLogic content that you need to retain for some reason.

RealAudio Music

Music was Real’s mono audio codec. It’s ancient and bad; RealAudio 10 or at least 8 should be used instead.

Stereo Music

The pre-RA8 Stereo Music codec is also no longer useful. RA8 offers superior efficiency and a much broader range of data rates.

RealVideo Encoding Tools

The Real compression tools and SDKs haven’t been updated for ages, but still work. Beyond the first-party products, many long-lived compression tools continue to include RealMedia support, although it’s not supported in newer products.

RealProducer Basic

RealProducer Basic is the free version of the RealVideo encoder, available for Windows and Linux. It’s easy for novices to use, but it doesn’t support fine tweaking of encoding settings; even SureStreaming is limited to just two streams. Professional results require Helix Producer Plus.

Real Producer Plus

RealProducer Plus is the paid version of RealProducer. Beyond all the necessary quality tuning controls, it adds a very useful bandwidth simulator feature to RealPlayer (Figure 20.2). This allows you to simulate what the stream would look like at different actual data and packet loss rates, and see what the lower band streams look like.

As a compression tool, it’s pretty basic, and hasn’t seen a significant upgrade in a long time.

Figure 20.2 The RealProducer bandwidth simulator is an easy way to test streaming performance under adverse network conditions, including packet loss.

Carbon

Of my daily tools, Carbon has the fullest RealMedia implementation (Figure 20.3), with the exception of Producer Plus’s Fast/Normal/Slow speed/quality control.

Figure 20.3 Carbon Coder has quite a complete RealMedia implementation.

Easy RealMedia Producer

The most full-featured RealMedia encoder that I know of is the freeware Easy RealMedia Producer (Figure 20.4). It’s a simple GUI on top of the RealMedia Helix SDK. It exposes all the knobs of that, and so has many controls even beyond those in ProducerPlus. This includes fine-grained control over complexity (it can go to 100, where the “High” setting in Producer Plus maps to 85), turn on B-frames, force all frames to be encoded, and other neat tricks. It’s really impressive; I wish I’d had it back when RealMedia was the bread and butter of my business.

Figure 20.4 Easy RealMedia Producer exposes advanced codec settings added for RealVideo after the last official SDK release.