Skip Navigation and You

Who needs a tagline when you have us?

Video: the last frontier of media standardization

August 27th, 2010 by Bob Crisler

The battle over online video standards has been going on for almost twenty years now. Apple demonstrated QuickTime in late 1991, showing the famous Ridley “Bladerunner” Scott-directed ’1984′ advertisement that introduced the Macintosh in a puny little postage-stamp-sized window on a Mac screen. I remember being astounded by that little video; the very idea that a computer could play video, however unimpressive it looked or sounded, was nothing less than a breakthrough in 1991. So I played that video and several others — “Sagittal Head” and “Saturn 5 launch” over and over — and was dazzled.

A year or so later, Microsoft Windows got similar technology; so similar, in fact, that it turned out that a lot of it was cut-and-pasted QuickTime code. The resulting litigation was not settled until 1997, when Bill Gates famously appeared above a beleaguered Apple conference, as Orwellian in appearance as the screen-bound antagonist from the ’1984′ ad, and announced MS’s continuing commitment to producing the Mac version of MS Office, as well as a $150M investment in Apple, overshadowing as symbol (a public vote of confidence in the company) the substance in the settlement of the first major skirmish on digital video patents.

What isn’t well-known to this day is that hidden at the center of the public rapprochement between Apple and Microsoft was a thorny issue that persists: who will control the digital formats on which video content is delivered? Who owns the video format?

There have been many skirmishes in the online-video format wars: the Rise and Fall of Real, the brief reign of The Three Formats (you had to encode Real, QuickTime and Windows to make sure all of your bases were covered), followed by the Five Years of FLV. Proprietary formats were battling it out, corporations chasing the dream of ultimately becoming the toll-taker for all digital video. And while every other media type (images, audio, etc.) settled on a set of standards, video did not, and the web suffered. Finally, Adobe’s widespread Flash plug-in gained the ability to play back h.264 video. And web developers in droves began encoding and delivering their content through that format. Of particular relevance to universities, the inclusion of h.264 support in Flash meant that the very same video file that was being played online in a web page could be uploaded to iTunes U.

Things seemed settled. So, naturally, it’s time for a flashback.

In the late nineties, the Motion Picture Experts Group, the part of the International Standards Organization that ratifies transport and encoding formats so that software developers, producers and hardware manufacturers can build markets around those standards (think CDs and DVDs for starters) … that group, MPEG, wanted to develop a new, better standard for digital audio and video. It would be called MPEG-4. So the Motion Picture Experts Group asked for submissions of then-current container architectures (container architectures differ from codecs; container architectures typically support many codecs) to be evaluated as candidates for the new standard. The short of it is that Apple’s QuickTime format became the MPEG-4 container format.

The seed of a new, global standard for online video was forming. It would be ratified by the same organization that brought us MP3 audio and the MPEG-2 video format that is the basis for the video content on every DVD in existence. To say that the MPEG standards had credibility is to vastly understate the case. The MPEG standards were digital media; they formed the basis of several multi-billion-dollar industries.

MPEG-4 would be aimed squarely at solving the problem of proprietary video and audio standards online. But it wasn’t until the high-quality video codec variously called h.264, MPEG-4 Part 10, or the Advanced Video Codec (AVC) was introduced as a component within the MPEG-4 container that things got really interesting. It was a great codec, better than anything else. Good enough that it gained a significant following as the standard to beat, solidifying itself as the de facto data standard for video on Blu-Ray, satellite and cable. But all of those formats are products. They generate income for the hardware manufacturers, the cable and satellite companies, and the developers of production suites. There’s money being made, and none of those companies think twice about paying for using the MPEG patents.

A patent pool was formed, now 27 members strong. Apple is there, of course, as is Microsoft. Sony, Sharp, Dolby Labs and every other consumer electronics company you can think of. And one of our higher-ed cousins, Columbia U. Each of those entities developed part or parts, plural, of the h.264 standard. And each of them has employees who expect to be provided food, clothing and shelter. They expect to be paid for their ideas, ideas that are expressed in the code that forms the h.264 codec.

But the online world was, and is, different. Much of the video being distributed online is being produced and distributed by nonprofits, or by individuals just for the fun of it. Paying MPEG-LA, the company taking payment for the patent pools for MPEG-2 and MPEG-4, for encoding in their format, while realizing no income for the video being delivered, Just Wasn’t Gonna Work.

Once again, there would be conflict. Free software and open standards had been gaining steam, beginning in the late nineties. A committed and zealously-held point of view proclaimed the argument that ideas should be free, content should be free, or at least content creators should be able to choose to make their content free, and that encapsulating that content inside a patent-encumbered format ultimately represented a compromise on that ideal. So a lot of ultimately pointless work was done to try to make the ancient underpinnings of On2 Technologies’ abandoned VP3 video format competitive, released in its second incarnation as the free-and-open Ogg Theora codec.

Meanwhile, Adobe added h.264 support to Flash, and Apple put out a phone that played video, but only h.264 video. On that phone on the day of its announcement was a YouTube app that played h.264 video, which sounded a bit odd at the time since YouTube video was all encoded in On2 Technologies’ VP6 delivered within a Flash container (FLV). But, on cue, at the iPhone unveiling a YouTube representative made the announcement that h.264 would become the base format for video on the YouTube, and that they’d begun converting their library to the new format. It was a big deal for creating a unified video standard; a very big deal.

On the opposing ‘open video’ front, despite high-profile support from Wikipedia, the Theora codec in all likelihood died this spring when Google bought, then later open-sourced, On2 Technologies’ VP8 (five better than VP3!) codec, renaming it WebM. For a while, it looked like WebM might upset what had been a solidifying ecosystem of support for h.264 video online. WebM appeared to be high-quality, a true competitor to h.264. Many would say it was a dead-even wash in quality, giving WebM the advantage in that it was, and was to remain, free and open.

But wait a minute, people cried. Even though the web development community generally likes and trusts Google, the same couldn’t necessarily be said about On2 Technologies. Some experts even went so far as to dig into the source code in VP8/WebM. Source code generally is the level where patents are examined, and enforced, in software. Remember that settlement between Microsoft and Apple regarding code that was copied from QuickTime and ended up in VIdeo for Windows? History, as they say, tends to repeat itself. The bugaboo for free formats that solve very complex problems is submarine patents; the only way to make absolutely sure that developers don’t copy code from somewhere else is to send them to live on a sister planet, light years away, while they code. And that, even if Ridley Scott’s directing it, is a current impossibility.

As it turns out, the developer of the free x264 h.264 encoder (if I’m losing you now, keep going … you’ve got too much time invested to stop now), Jason Garret-Glaser, went looking. And what he found was a veritable minefield of potential submarine patent claims in VP8/WebM. “With regard to patents, VP8 copies too much from H.264 for comfort, no matter whose word is behind the claim of being patent-free. This doesn’t mean that it’s sure to be covered by patents, but until Google can give us evidence as to why it isn’t, I would be cautious.” That’s the way he put it. The way I’d put it is “Here, There be Dragons.”

Yesterday, MPEG-LA, the organization that administers the MPEG-4 patent pool, announced that h.264, AVC, Part 10, whatever you want to call it, would be permanently royalty-free online as long as the video is free to end users. In other words, if you encode your video in h.264 and put it on the Internet to be enjoyed by others, and you don’t charge those viewers to view your content, neither will the patent-holders of h.264 charge you for using their patents. And those patents are known, legally-vetted, and in contrast to WebM, much closer to being a sure thing. Granted, h.264 isn’t ‘free,’ as in ‘freedom’ in the sense that technogurus like Richard Stallman espouse.

But it’s free as in money, forever, and that’ll be good enough for most.

Leave a Reply