Introduction

It was the fall of 1989, and I was taking a computer animation class at UMass Amherst (I went to Hampshire College, another school in the Five Colleges consortium). We were using long-gone tools like Paracomp Swivel 3D to create a video about robots. This was well before After Effects, so we were using Macromind Director 1.0 as a very basic compositing application. We needed faster playback on our 20 MHz Macintosh IIcx to preview animation timing. I started experimenting with a utility called Director Accelerator, which composited together all layers of the Director project and compressed the final frames with Run-Length Encoding (RLE), a very early form of compression. I was amazed how the resulting files were smaller than the originals. And so I started playing around with optimizing for RLE, and forgot about animation and robots.

I got a D in the class, while the TAs and some other students went on to create Infini-D, a pioneering 3D app for desktop computers. Infini-D lives on as Carrara from DAZ. As for me, I didn’t do any animation after that, but darn if that compression stuff didn’t stay interesting.

So interesting I’ve made a career of it, although it took me a while to recognize my destiny. Right after college, I spent a year writing and producing a comedy-horror mini-series with my friends. I wrote an overly long scene that hinged on the compression ratio of Apple Compact Video (later known as Cinepak). I was sure it was interesting! My collaborators never quite believed an audience would enjoy three pages of innuendo-laced banter about kilobits per seconds.

Next, I cofounded a video postproduction company Journeyman Digital in 1994. We started trying to be a standard NLE shop, using a Radius VideoVision Studio card in a PowerMac 8100/80, with a whopping 4 GB RAID drive ($5,000 for the drive, another $1,000 for the controller card). However, due to bugs in that generation Mac motherboard, we couldn’t capture more than 90 seconds in sync, making the whole system a better boat anchor than video editor.

Then, one day, someone called us up and asked if we could do video for CD-ROM. They were willing to pay for it, so we said “Yes!” and dove in, trying to figure out what the heck they were talking about. It turned out the VideoVision was perfect for short CD-ROM clips, which never had good sync anyway.

For compression, we used Adobe Premiere 3.0 and Apple’s MovieShop. The codecs of choice were Cinepak and Indeo 3.2, with 8-bit uncompressed audio. Data rates for a typical 240 × 180 15 fps movie were around 120 KBytes/sec, more than enough for full-screen web video today. We targeted 80 minutes of encoding time per minute of output, so there was a lot of “bartending.”

Still, primitive as the technology was, it enabled some amazing things. Today, that computers can play video is a given, so it’s hard to convey the thrill of seeing those first postage-stamp videos play in a small rectangle on the screen. The video might have been small and blocky, but it was interactive! We could actually add video to programs. We worked on all kinds of sales training discs, kiosks, encyclopedias, and multimedia training projects for fun and profit.

By the late 1990s, video on the web was all the rage. Better, faster, cheaper CPUs and Internet connections and new web technologies finally delivered web video that looked like more than pixel soup. The future looked promising. New businesses were springing up like dandelions—everybody wanted to cash in on the opportunities presented by the coming of ubiquitous broadband connectivity. Companies like Akamai, Digital Island, and many others set up content delivery networks and raised billions in their initial public stock offerings.

It was in those heady days I met my future wife, Sacha, at the Portland Creative Conference. I didn’t know it when we first met, but she’d been doing digital video about six months longer than I had, and remains the only known person to have made money with the infamously unfinished SuperMac VideoSpigot capture card.

In 1999, I joined Terran Interactive, the original developers of Media Cleaner Pro (the leading professional video compression application at the time) to launch and run Terran’s Consulting division. My job was to reap services revenue and to help with marketing and product development. Media 100 went on to acquire Terran, and I was laid off with most of the ex-Terrans when the big Internet bubble “popped” in 2001.

Around this time, we had our first child, and really wanted to stay in Portland to be near family. The local media companies were downsizing, so my wife and I put out our own shingle as a consulting and media services company called Interframe Media. It was a fruitful period, as we expanded our focus beyond compression services to compression tools, and I consulted on the design of many different products, including Windows Media Encoder, Adobe Media Encoder, Telestream’s Episode Pro, Canopus ProCoder, Rhozet Carbon Coder, and Sorenson Squeeze.

While things seemed dire in 2001, and a lot of companies went under, there was a reason there had been so much money flowing into the industry. Delivering compressed video over the Internet was going to be big someday. And even if many investors spent too much, too early, that effort laid the foundation for today’s industry, and—in an important change—some profitable businesses as well!

Interframe Media had five years of steady growth, and we were trying to figure out if we should actually hire some employees to take on the big projects we were already turning down. As we entered the summer of 2005, the emergence of the HD DVD and Blu-ray war and our third child made us realize that between household and business, we simply didn’t have enough time to take on this next generation.

That fall, Microsoft, Amazon, and Google all were recruiting me for their various digital video efforts. All three opportunities were exciting and flattering, but in the end I chose Microsoft, to the surprise of those who’d known me as a Mac user of two decades. But Microsoft had long been at the center of digital media, and in Windows Media 9 Series delivered the first really complete media delivery technology that scaled from phones to HD. After stints on the HD DVD and codec teams, I wound up as the media strategist for Silverlight, working on the whole end-to-end ecosystem of publishing video content. In its own way, that’s what made it possible to do a second edition of the book—the format wars were over. Silverlight has joined Flash, QuickTime, and Windows in broad format support, including H.264. The issue of how and what codec you compress to isn’t nearly as tied to where it gets played back as it used to be.

The world of compression has exploded in the 20 years since those first lurching frames on a CRT. With the 2009 analog television switch-off in the United States, we’re seeing the end of video that isn’t digitally compressed. Compression’s a fact of life for almost everything we see or hear, whether it’s delivered via an MP3, DVD, HD broadcast, kiosk in a store, video clip on a phone, voices on a phone video on demand over cable…the list goes on. And while we once fantasized about achieving “VHS quality” on the Web, delivering HD video that looks as good as broadcast and cable is now commonplace.

This book has been a long time in the making. I wrote the outline for the first edition back in 1998, when CD-ROM video was all the rage and web video something we knew would happen “someday.” When it was finally published (after one wedding and two baby-induced delays) in 2002, the overall goal hadn’t changed a bit, although that someday had become a now. It was all about how to make good compressed video on the many delivery platforms available.

After it had gone to press, I immediately had grand plans for a second edition, but between business, family, and jobs, it always seemed like something I’d get to “in a few months.” But, as in most big things in life, when you realize there’s never going to be a perfect time, it’s probably the right time to do it anyway. In the end, I was inspired by those who kept reading, discussing, and even buying this half-decade-old book. I owed them a new edition for the compression universe of today.

And it really is a new world now. The first edition covered dozens of different codecs and nearly as many media formats and platforms. We’ve seen a great convergence recently, with the standardized codecs of MPEG-2, VC-1, and H.264 dominating most new content and players. Being a compressionist today allows for more depth, as we don’t have to use and master the myriad and varied tools and technologies once required. Conversely, we’re seeing much more variety in the places we can play that content back; the first edition was mainly about content that would play back in a particular media player or a browser window on a Mac or Windows PC. But the devices and services available today are much, much more complex, while at the same time the demand for “any content, anywhere, any time” is only increasing.

Who Should Read this Book

This book is for compressionists, people who want to be compressionists, and people who on occasion need to pretend they’re compressionists. Most of you will have come from the worlds of video or the web. Video people will be happy to hear that this stuff isn’t as alien as it might initially seem. The core skills of producing good video are just as important when dealing with compression, although compression adds plenty of new twists and acronyms. Web folks can find the video world intimidating—it’s populated by folks with 20-plus years of experience, and comes festooned with enough jargon to fill a separate volume. And there are plenty of people who just need to get some video compressed, fast, coming from entirely different fields.

That’s fine; this book is designed to help you get to where you need to be, regardless of where you’re coming from. To that end, I’ve tried to include definitions and explanations of this jargon as it comes up, and gently introduce concepts from first principles. If you read this book in nonlinear fashion and jump straight to the problem you need to solve today, there’s a glossary that should help you decode some of the more common terms that crop up throughout.

How This Book is Organized

The chapters are organized in three main sections. The first section (Chapters 1–3) covers general principles of vision, compression, and how compressed video operates. While a fair amount of it will be old hat for practicing compressionists, there should be something of interest for everyone. Even if you skim the first time around, it’d be a fine idea to go back and give it a more careful reading sometime while you’re bartending. I’ve tried to keep it readable—there’s no math beyond basic algebra, and I’ve kept it to details that matter for compression. Since this is “evergreen” stuff for the most part, readers of the first edition will find a lot familiar (and a lot new as well).

Chapters 4–8 cover the fundamentals of video technology and the non-codec parts of the compression workflow. This includes compression-friendly production techniques, video capture, and preprocessing.

Chapters 9–29 cover specific video tools and technologies. Each chapter is self-contained, so it’s fine to skip straight to the ones you care about. I include many scenario specific tutorials illustrating how to solve common problems and also how to approach new projects with forethought and experimentation.

The world of compression changes fast enough to give you chronic whiplash. Given the breadth of the topic, I was constantly updating chapters to keep up with new tools and technologies. But we had to stop at some point. Even if the book is a version behind the current release of something, I try to explain the why as much of the how in order to make the skills and mental approach applicable to future versions as well as entirely new tools.

Check http://www.benwaggoner.com/bookupdates.html and http://www.cmpbooks.com/compression for updates, corrections, and other resources.

Acknowledgments from the First Edition

A book like this requires a lot of work. Dominic Milano went far above and beyond the call of duty as technical editor, making sure it was readable, relevant, and right. Jim Feeley edited many articles for DV that were later incorporated here and continues to teach me an enormous amount about how to express complex technical ideas clearly. The long course of this book provided many opportunities to demonstrate the patience, talent, and good humor of Dorothy Cox, Paul Temme, and Michelle O’Neal from CMP Books.

Lots of people taught me the things I said in this book. The founders of Terran Interactive, Darren Giles, Dana LoPiccolo-Giles, and John Geyer answered many dumb questions from me, until I knew enough to ask some smart ones. I learned a lot about compression from them via late night emails.

During the gestation of this one book, my wife Sacha gestated Alexander and Aurora—a much harder job than mine! I’m looking forward to spending more time with them instead of typing in the basement.

My family helped me start a business back when I didn’t know enough about what I was doing to even know what I didn’t know. I’m sure they bit their lips more than they let on over the past few years. Hopefully, they’re able to relax now.

And finally, none of this would have happened without Halstead York, who over the years has talked me into writing scripts, producing video, starting two companies, and learning compression. Without his infectious enthusiasm, vision, and drive, I’d probably still be writing banking software.

Acknowledgments from the Second Edition

Sacha, Alexander, Aurora, Charles

SteveSK, TimHa, AlexZam

Wei-ge Chen for DCT visualizer

Phil Garrett for Microsoft Viewer