CHAPTER 15
FLV

Flash Video—FLV—was where Flash came of age as a media platform, not just an animation plug-in. Flash started as FutureSplash Animator back in 1996, just doing simple vector graphics animation. Its potential for media was seen early on—I first saw it in the keynote of the RealNetworks conference that year. After several years of kludgy third-party technologies for video-in-Flash, Macromedia licensed the Spark codec from Sorenson Media, introduced the FLV format, and built them into Flash 6 player in 2002. This worked well as a way to add video to Flash presentations, but compression efficiency wasn’t sufficient for video-centric projects. The much-improved VP6 video codec was added in Flash 8 in 2005, and found widespread use in many high-profile sites.

Adobe added support for standards-based MPEG-4 files with H.264 and AAC audio in a Flash 9 update, and much of the Flash ecosystem has been moving to that for broader encoder support, increased efficiency, and interoperability. See Chapters 11 and 26 for more details.

Still, FLV offers some unique features and advantages, like alpha channels, easier decoding, and backwards compatibility, and remains widely used. Adobe has helped this effort by releasing a free specification for the FLV file format itself.

Why FLV?

FLV is increasingly becoming a legacy format, but it does have its places. The “default” encoding for Flash is now MPEG-4 H.264 (“F4V”), but there are still good reasons to go to FLV in some cases, as described in the following sections.

Compatibility with Older Versions of Flash

While most consumers upgrade to new versions of Flash pretty quickly, corporate users require IT assistance to update their Flash players, and so they lag behind. If your market contains a significant number of users behind the firewall, FLV will offer greater backwards compatibility.

Decoder Performance

Both Spark and VP6 require less CPU per pixel than H.264, allowing higher frame sizes on older machines. Spark in particular is very simple and thus very fast. And VP6 can dynamically reduce or turn off its postprocessing to offer stable frame rates on older hardware (with reduced video quality, of course). There’s also the VP6-S simplified mode, which reduces decode complexity another 20 percent or so, albeit with a slight reduction in compression efficiency.

That said, there’s also flexibility in H.264 to tune for simpler decoding. If decoder performance is a primary concern, you can try making H.264 with the following settings:

The first is automatic with Baseline Profile, so that’s easy to try. These wouldn’t be as efficient as a full-bore High Profile, but certainly is still more efficient than VP6 and especially Spark. Even without in-loop deblocking, a good H.264 Baseline would dramatically outperform Spark in coding efficiency, but may not outperform VP6 with its rather advanced postprocessing filter.

Alpha Channels

FLV is the only way so far to get alpha channel video in Flash, supported in both Spark and VP6. While there are alpha channel modes for H.264 and MPEG-4, Adobe doesn’t support them as of Flash 10.

Why Not FLV?

Flash Only

The biggest drawback to FLV is that it’s a Flash-only file. An .f4v is just an .mp4 (and can be renamed as such), and thus is already compatible with QuickTime, Silverlight, and many other media players.

There are third-party players that include FLV playback, like VLC, but out of the box QuickTime and Windows Media Player do not. And both OSes have had better in-box codecs for many years.

Lower Compression Efficiency

VP6 and especially Spark need more bits for the same quality level as H.264. If you’re more limited by bandwidth than decoder performance, a well-tuned H.264 can be a dramatic improvement.

Fewer and More Expensive Professional Tools for VP6

Spark is just H.263, so it’s cheap and readily available, even in open source tools like ffmpeg.

VP6, however, is expensive to license, particularly for the full-featured versions and incredibly so for enterprise licenses. This is why some high-end tools like Carbon just ship with the very limited 1-pass only QuickTime Export Component encoder from Adobe. Telestream is the only vendor to bundle the full VP6 across most of their product line.

Sorenson Spark (H.263)

The Spark video codec is an implementation of the standard H.263 videoconferencing codec. It was introduced with Flash 6, along with FLV. H.263 is a lightweight codec, supported on all kinds of devices, and fast to decode.

Given all of this, Spark can offer decent quality at higher bitrates, particularly when using 2-pass VBR encoding.

Not all Spark versions are the same. The implementation in Squeeze has many options (Figure 15.1), while the one in Adobe Media Encoder (Figure 15.2) really just does bitrate and Key Frame Distance. Telestream’s implementation in Episode is based on their own MPEG-4 part 2 implementation (Figure 15.3).

Figure 15.1 Sorenson invented Spark, and still has the deepest implementation of H.263 for FLV.

Figure 15.2 Adobe Media Encoder’s H.263 support is a very limited 1-pass CBR only.

Figure 15.3 Episode’s H.263 is a subset of its MPEG-4 part 2, including Telestream’s unique Lookahead-based 2-pass mode. It has no control over peak bitrate, just buffer duration.

Quick Compress

Quick Compress is a faster, lower-quality mode. Leave it off unless you’re in a huge rush; Spark is very fast regardless.

Minimum Quality

Minimum Quality sets a minimum visual quality (presumably mapping to maximum QP), below which the codec won’t drop. Instead, it raises the bitrate of each frame until it hits that target. With Drop Frames turned off, the Minimum Quality feature can cause the data rate to overshoot the target substantially. Turn Drop Frames on if you must use Minimum Quality and don’t want to overshoot the target. Minimum Quality should generally be off unless legibility of elements on the screen is critically important.

Drop frames

If this option is on and you’re encoding in CBR mode, the frame rate will drop to maintain data rate. This option doesn’t do anything in 2-pass VBR encoding. You should have it on in CBR mode, even if you turn Minimum Quality down to 0. All CBR encodes should at least have Minimum Quality = 0 and Drop Frames on to ensure output really has a constant bitrate.

Automatic Keyframes

This feature lets the codec start a new GOP at scene changes and high motion. It should always be on. The slider controls how sensitive the codec is to keyframes. The default of 50 is good for most content.

Image Smoothing

Image Smoothing is a flag telling the decoder to apply a deblocking postprocessing filter. This helps improve quality of compressed frames. It does increase decode complexity somewhat, so shouldn’t be used when targeting very slow playback systems with higher data rates and resolutions, but most of the time should be used. Spark is a very fast decoder in general.

Playback Scalability

With the playback scalability option, the video can drop the frame rate in half instead of presenting uneven stuttering on slower machines. Playback scalability uses the scheme from the ancient Sorenson Video 2, with even and odd video frames encoded as two parallel separate streams. Scalability is achieved by simply not playing one of the tracks. This reduces compression efficiency, because it doubles the temporal distance between frames, making motion estimation much less efficient. It also makes decoding both streams somewhat slower than just doing a single stream.

Because of these limitations, Playback scalability hurts more than it helps and shouldn’t be used.

On2 VP6

The big video upgrade in Flash 8 was On2’s VP6 codec. On2 (formerly the Duck Corporation) has been engineering codecs for well over a decade now, and VP6 offered a good combination of compression efficiency and easy decoding. VP6 is also used in JavaFX, but not in a FLV wrapper. On2’s VP7 and VP8 are better codecs yet, but Adobe has since switched to H.264 as its primary high-efficiency codec for Flash, and hasn’t added support for the more recent VPx decoders. On2 supports MPEG-4 in their Flix products now as well. Google announced their aquisition of On2 in 2009. It remains to be seen what if any impact this would have on further development of VP series of codecs.

VP6 offers a pretty massive improvement in compression efficiency over Spark—you can get away with half the bitrate in some cases. Decode is somewhat slower, although VP6 varies postprocessing quality based on available CPU power, so it can vary quite a bit.

The biggest drawback to VP6 is that it’s pretty slow compared to other modern codecs; it’s still single-threaded in its most common implementation (although that’s changing; see “New VP6 Implementation” later in this chapter).

In some ways, postprocessing is the most impressive aspect of VP6. Beyond normal deblocking (which it does quite well), VP6 even will add grain texture to the video to mask the loss of detail from compression, making it seem quite a bit more detailed than the retained details would otherwise provide.

Alpha Channel

VP6 also: supports alpha channels. If you have alpha-channeled source (like from After Effects), you can encode that alpha in the movie file for real-time compositing on playback.

Flash doesn’t support alpha channels in F4V, so FLV is the only way to provide that today.

Figure 15.4 The incredibly complete and long Squeeze VP6 dialog. I love having all my parameters in one pane, but should it really require a 2560 × 1600 display?

Figure 15.5 Adobe Media Encoder’s VP6 support has 2-pass and VBR, but not a lot of other parameters.

VP6-S

VP6-S is a newer variant of VP6 that constrains decode complexity somewhat while remaining backward-compatible. It does lose some encoding efficiency as well, but the playback improvement is bigger, so it can increase maximum quality when CPU is the limit, not the network.

New VP6 Implementation

On2 has been working on an updated VP6, promising improved quality and (finally!) multi-threading for much improved encoding performance on modern hardware. These are initially available in their Enterprise products, but should eventually migrate down to their Pro products.

VP6 Options

VP6 offers a wide variety of options in its Pro version, as supported in Sorenson Squeeze (with installation of the “On2 VP6 for Adobe Flash” add-on), in On2’s own Flix products, and any product using QuickTime Export Components with the “Flix Exporter” installed.

Figure 15.6 Episode’s VP6 setting. Tuned for mobile, it also exposes error resiliency.

There’s also the much more basic Adobe version, implemented in Adobe Media Encoder and the “Flash Video” export component that’s installed along with the Flash products. That’s 1-pass CBR only and lacks all of the following advanced settings. Use Adobe Media Encoder at least, or if you need QuickTime export integration, buy the Flix export component.

The advanced modes definitely pay off in improved quality and efficiency for almost every project. If you’ve got a good reason to use FLV, you should get access to the full version.

The Sorenson and Flix products expose a slightly different set of controls, so I’ll discuss both here.

CBR vs. VBR

In general, you should use CBR when you’ll be delivering video using Flash Media Server, and VBR the rest of the time. The 2-pass modes for both CBR and VBR give higher quality than the 1-pass modes, albeit at twice the encoding time. VBR in particular benefits from 2-pass encoding.

Compress alpha data

This applies some mild lossy compression to the alpha channels. The default is generally okay, but reduce this compression if you’re having trouble getting a clean edge.

Figure 15.7 The Flix VP6 dialogs. The Advanced Features (15.7B) have some of the most esoteric terminology in the industry.

Auto key frames enabled/key frame settings

In Squeeze, this turns on automatic keyframes. That’s quite important for quality, and should always be used for any content with scene changes.

Figure 15.8 Adobe’s QuickTime exporter. 1-pass only.

In Flix, turning Key Frame Settings to Fixed turns off automatic keyframes.

Auto key frame threshold

This specifies how different two frames need to be to trigger a keyframe. The default is almost always fine; you’d only change if you were seeing keyframe flashing.

Minimum distance to keyframe

Sets the maximum number of frames before the next keyframe. 1 is keyframe-only.

Compression speed

Best is the highest quality, but also the slowest. Good is faster, but a little lower quality. Speed is the fastest, but lowest quality. Use the highest quality you have time for, and be thankful if you have the multithreaded version.

Minimum quality/override quantizers: maximum

This is the maximum quantizer the encoder can use, and hence represents how much compression it can apply to a given frame. Higher values will lead to more dropped frames, but with a higher minimum quality for frames. I generally set to 0 (Squeeze) or 56 (VP6) to avoid dropped frames, and use other optimizations to achieve target quality.

Maximum quality/override quantizers: minimum

Maximum quality specifies the minimum quantizer for a particular frame, and hence the maximum quality and least compression. Using a lower value can preserve more bits for the more difficult portions of the video. The default maximum of 4 is fine for most web work, but a lower value may improve detail a bit if you’ve got bits to burn.

Drop frames to maintain data rate

Lets the encoder drop frames if there aren’t enough bits to do all frames at the minimum quality. If off, the data rate will just go up when there aren’t enough bits, and so this should be used with CBR.

Drop frames watermark/temporal resampling

This oddly named parameter has nothing to do with watermarks. It sets the percentage that the buffer goes down to before frames start getting dropped. It’s much more applicable to live and 1-pass CBR encoding. Default of 20 is a fine start.

Sharpness

Sharpness tells the encoder to emphasize a smooth low-artifact image rather than a sharper but potentially artifacted image. It won’t add sharpness beyond what’s in the source. Generally, the highest value that doesn’t introduce artifacts should be used; the more challenging the encode, the lower Sharpness will be optimal.

Noise pre-processing level

This activates the codec’s own internal noise reduction. Normally noise reduction should be done in preprocessing; if that’s not available, this mode seems to work reasonably well for sources with visible noise. Higher levels can make for a soft image, but can dramatically improve compression efficiency.

CBR settings

The next four options apply to CBR encoding only.

Maximum data rate/peak bitrate

This specifies how much of the available bandwidth the codec will try to use. While 100 would be the obvious choice, VP6’s rate control has historically not been very accurate, so using 90–95 leaves some headroom. 100 can be used with 2-pass CBR.

Starting buffer level/prebuffer

This specifies how many seconds of buffer are used at the start of the file. Higher values help quality, but delay start of playback.

Optimal buffer level

This denotes the buffer level the encoder tries to maintain throughout the file after it starts. It is generally higher than the Starting Buffer Level.

Maximum buffer size

The maximum length allowed for the buffer if a complexity spike pushes past the Optimal Buffer value. The codec will drop frames to keep within Maximum Buffer.

VBR settings

The following settings are VBR-only. Flix calls this “Two-Pass Section Datarate” even though it applies to 1-pass VBR as well.

Maximum 2-pass VBR data rate/max section (as % of target)

This determines how high the data rate can go. Higher peaks make for higher decode requirements, so the lowest value that provides adequate quality in the hardest sections should be used. 200 percent is a good target for HD encodes; the lower the bitrate, the higher the variability can be without risking decoder performance issues.

Minimum 2-pass VBR data rate/min section (as % of target)

This specifies the minimum bitrate to be used. Like MPEG-2, VP6 can require a minimum bitrate ensuring VBR algorithms don’t bit-starve very easy sections of the video. The default of 40 percent seems to work, but lower values can preserve more bits for other sections of the video.

VBR variability

This specifies how much the data rate can go up and down in a VBR encode. 0 is the same as a CBR. It seems like this should be bound by the Min/Max VBR bitrates, but it’s exposed as a parameter. Lacking any good data as to what it’s doing internally, I stick with the default of 70.

Data rate undershoot/undershoot % target

This sets a percentage of the target data rate to shoot for, to leave a little headroom in case you really can’t afford to go over. This is most useful in 1-pass encoding, to leave a bit reserve in case of very difficult sections.

FLV Audio Codecs

Almost all FLV files use MP3, but there are some other options that used to be available.

MP3

MP3 is by far the best general-purpose audio codec in FLV, and by far the most used.

Note that some very old versions of Flash had trouble playing back MP3 above 160 Kbps. If you’re encoding Spark FLV for backward compatibility, you may want to stick to that limit.

Remember to encode MP3 in mono at lower bitrates; it’s more important to keep the sample rate high than to have two channels. Generally, content delivered at 64 Kbps and below should be mono.

Nellymoser/Speech The

Flash encoder used to include a licensed version of the Nellymoser Asao codec, called “Speech” or “Nellymoser” depending on the version. Like the other examples of its genre, Speech is suited to low-bitrate, low-fidelity speech content. MP3 does a fine job with speech and can provide better compression efficiency, so Nellymoser Speech was only useful at very low bitrates. Encoding it isn’t supported in the current tools, for which my largest regret is not being able to say “Nellymoser” during classes.

ADPCM

ADPCM is a very old 4:1 compressed audio codec that’s supported in old versions of Flash. You’d never encode with it now, but it may be found in some very old files.

PCM

The PCM audio codec is straight-up uncompressed audio. It would be very unusual to use it in a FLV. Authoring tools include it because older versions of the Flash application would automatically re-encode all the audio imported in the project.

FLV Tools

A variety of tools offer FLV support, although they can vary widely.

Some tools are dropping built-in support for FLV now that Flash supports H.264. But any tool that supports QuickTime Export Components can encode to FLV using either the Adobe or (much better) On2 implementations.

Adobe Media Encoder CS4

Media Encoder CS4 has basic FLV encoding, tuned for simplicity over configurability, as is the case for most of its settings. It has added 2-pass and VBR modes, a good improvement over previous versions.

QuickTime Export Component

Adobe used to bundle a QuickTime Export Component with Flash that lets QuickTime-based apps encode to FLV. It’s very limited, unfortunately, providing 1-pass CBR only and few other configuration options. It is no longer installed as of CS4.

Flix

Wildform pioneered the video-in-Flash market with Flix, a system that provided limited video functionality to Flash prior to Flash 6 and FLV. On2 purchased Wildform’s Flix products and has turned them into pretty rich encoding products for FLV and MPEG-4, with good integrated skin support.

There are three Flix products: the flagship Flix Pro (Mac and Windows), the more limited Flix Standard (Windows-only), and the Flix Exporter (Mac and Windows) QuickTime Export Component. Note that the $199 desktop version of the Exporter is limited to 1500 files encoded per month; more than that requires the $3,000 “Server Edition.” They’re otherwise identical. $3,000 is an unprecedentedly high price for a codec, particularly one that isn’t multithreaded. That pricing has driven a lot of F4V adoption.

Most professionals would use either Pro or Exporter—Standard lacks basic pro features like 2-pass VBR and batch encoding, and really targets the amateur market.

There are also SDK versions of Flix used to build custom workflows and implement enterprise encoding.

Telestream Flip4Factory and Episode

Telestream’s FlipFactory 6 is the only enterprise encoding tool that comes with the full Server Edition for VP6, offering all the parameters in Flix and Squeeze. And their desktop Episode Pro (but not the non-Pro version) includes a full implementation as well. Unlike most of the other Episode codecs, it implements On2’s 2-pass rate control mode instead of Telestream’s unique Lookahead-based multipass.

H.263 FLV support in both products uses Telestream’s own implementation. They are the only products other than Squeeze to support the error resiliency feature of H.263 for improved quality over lossy networks, via the Intra Refresh Distance option.

Sorenson Squeeze

Sorenson Media was the creator of the original Spark codec for Flash, and also provides deep support for VP6 with a $199 codec pack (and, of course, makes great F4V files).

Squeeze exposes all the parameters Flix does; it’s your preference as to which encoder you want to use.

ffmpeg

The open source ffmpeg includes FLV support, based on its existing support for MPEG-4 Part 2 (H.263 is MPEG-4 part 2 Short Header). It doesn’t have VP6 encoding support.

I bet ffmpeg is responsible for more eyeball-hours of FLV encoding than anything else; that’s what YouTube used for its FLV files.

While ffmpeg front-ends don’t generally include FLV support, it’s not that hard to achieve via the command line. The example you’ll find by searching “ffmepg FLV encoding” is pretty much identical everywhere, cloning the old YouTube settings: –I “foo.avi” –acodec mp3 –ar 22050 –ab 32 –f flv –s 320 × 240 “foo.flv”

Alas, this replicates the horror of old YouTube audio at 22.05 kHz, and doesn’t even set a bitrate.

Much better sounding would be: ffmpeg –I “foo.avi” –acodec mp3 –ar 44100 –ab 64 –f flv -b 400–s 320 × 240 “foo.flv”

Which improves audio and sets the video bitrate to 400 kbps. There’s plenty of further tweaking that can be done to improve quality; with enough effort, this would probably match the highest-quality Spark files, but it won’t match VP6 Pro.