Mozilla Web Development

Debating Ogg Theora and H.264

Since the big HTML 5 news that there will be no defined codec for <audio/> or <video/> there has been a lot of discussion about the merits of such a decision, and what led to it. To quote Ian Hickson’s email:

Apple refuses to implement Ogg Theora in Quicktime by default (as used by Safari), citing lack of hardware support and an uncertain patent landscape.

Google has implemented H.264 and Ogg Theora in Chrome, but cannot provide the H.264 codec license to third-party distributors of Chromium, and have indicated a belief that Ogg Theora’s quality-per-bit is not yet suitable for the volume handled by YouTube.

Opera refuses to implement H.264, citing the obscene cost of the relevant patent licenses.

Mozilla refuses to implement H.264, as they would not be able to obtain a license that covers their downstream distributors.

Microsoft has not commented on their intent to support

I think everyone agrees this is going nowhere and isn’t likely to change in the near future. For the sake of moving HTML5 forward, this is likely the best decision.

Here’s how I interpret everyone’s position:

Apple’s Argument

One of the undeniable perks behind H.264 right now is that there is hardware decoding available and used on on certain devices. One of the most notable is the iPhone. Using hardware decoding means your not using the CPU which results in better performance, and most importantly better battery life.

Thus far there’s no hardware Theora decoder on the market (if you know of any let me know, my research says none), which I suspect is why Apple is hesitant to jump on board. Until there’s hardware that’s proven to perform well, be cost-effective in the quantities Apple needs, and not be bombarded with patent infringement claims, I suspect they’d rather settle with H.264. The patent part is critical. Apple can update software to comply with patent wars pretty quickly, as many other companies have done with software in the past. Hardware is not so easy. Last minute hardware changes are harder to deal with than software because of the many things it impacts, and the inability to update at a later date.

I’m almost positive the lack of hardware support is the exact same reason Apple has been so against Flash support. Remember the YouTube application isn’t using VP6 like regular flash, it’s using H.264 (that’s why it took so long for all of YouTube to be available on the iPhone).

If there’s enough Theora content out there, there will likely be Theora decoder hardware made to meet market demand. To get to this point will be difficult with the amount of VP6 (Flash) and H.264 content already on the web. H.264 alone has a major head start in applications. VP6 has several years of video on the web now (and I still don’t think it has a hardware decoder on the market though that might be due to licensing again).

In the long run, I think mobile technology will improve enough to make this a somewhat unnecessary constraint. Mobile CPU’s and GPU’s are just starting to get to the caliber needed for video. Performance per watt should improve. Battery technology is just starting to get pushed to the limits. This is a good thing for Theora in the long run, but the question is how long?

Until it can be played with minimal impact on battery life, I don’t think any company who has a heavy investment in mobile will want to jump on board.

Google’s Argument

Google has money and can license H.264. Shocker. Google however has trouble when it comes to Chromium. I suspect Google doesn’t care too much about which way this goes since what they support in Chrome doesn’t mandate that YouTube support it. However if the encoding quality for a given bitrate is good enough, it becomes a viable option.

Regarding the quality argument, I’ll simply point to this comparison. I the quality today is comparable already, and likely to get better as the encoders improve. I’ll leave this discussion here.

Opera’s Argument

Opera says H.264 is to expensive to license. I don’t know what the costs are, and what they would be for Opera, but I’ll take their word on it. After all, the do have a product available for free download. While commercial and closed source, they don’t have Google’s revenue stream and I respect that.

Mozilla’s Argument

Mozilla can’t license for downstream Gecko use etc. I’m sure a good part of the argument is also that requiring licensing fees to use <video/> is bad for the web and open source. I agree.

Microsoft’s Argument

No comment. Historically they implemented <marquee/> but not the <blink/>. Make of that what you will.

<video/> could be supported by plugin if needed. I recall Adobe supporting SVG by plugin a few years ago.

Where to go from here?

I think there are a few possible outcomes. As for what I think are the most likely:

  1. There’s a push for hardware decoding that makes Theora on mobile technically possible and working well. If Apple legally is satisfied and jumps on board that changes the game. As I stated earlier I think Google is mostly ambivalent since they support both right now. Opera doesn’t want H.264 anyway, so they are cool. IE 8 can likely be handled by a plugin. Apple really is the deciding factor. Theora is the future.
  2. See what the web does. I suspect at least for a long while the web will just stick with Flash since it works on almost all desktops. For mobile the iPhone and Android make up pretty much the bulk of the mobile video market and that doesn’t look like it’s changing so fast. Content providers that want mobile will encode for mobile. That means 3 target platforms, not ideal but reasonable. H.264 and whatever Adobe adopts is the future.

I know how the media is interpreting all of this. How do other developers, and open source folks see it?

18 replies on “Debating Ogg Theora and H.264”

Regarding “see what the web does”, I agree that “nothing” is the probably answer. Personally, I count three browsers with significant enough market share to worry about – IE, Firefox, and Safari. Of those, one doesn’t support video-tag at all, and the others don’t support the same codecs – the only solution that works on all three is the status quo, Flash-based players. You’ll see a few sites using one of the two video-tag variants, but not to any significant degree.

(No offense to those who favour Opera or Chrome, but I don’t think they count for much when it comes to which way the herd is going.)

I think the easiest way to handle this unless you want to have two videos for every video on your site is to recommend that people either install the Xiph Theora plugin if they are on a browser that doesn’t support it, or recommend Firefox.

If you can afford to have a Theora an h.264 version of every video, the fallback method is the best, and covers all platforms.

But you can definitely not afford to ignore the 300 million users on Firefox, that’s for sure — so merely supporting h.264 won’t fly.

As I understand it, most people mis-understand the hardware decoding thing. Turns out that you don’t make custom hardware for the various codecs. There’s a range of commodity hardware that you write some specific software for. Apple could, probably in short order, write this software for their two or three iPod, iPhone, and AppleTV chips and you’d have your “hardware decoding”. 

Or to put it another way, this isn’t really about hardware. It’s about software.

– A

Flash, Silverlight, Moonlight, Windows Media Player (on w7 ) , Quicktime, Safari, and Chrome do H.264, so that’s pretty much everywhere covered already.

Can’t you just redistribute/sublicense a H.264 decoder from MS or Apple or Google or someone who has already hit the maximum fee? MS already do this for Moonlight, and someone wrote a plugin that hosts Moonlight and injects it pretending to be WMP on linux… free as in beer VC-1 on Linux….

I wish you luck overcoming momentum.

@Asa Dotzler – That largely depends on which chips Apple have used. Some are hard-coded, others are reprogrammable (and even then it’s often only the manufacturer who can do it).

@Fred: Even if it were possible to sublicense H.264 this won’t save authors from the H.264 content-delivery fee (they’ll charge for content after 2010). This is completely unacceptable. Plus any H.264 license cannot be attached to source code, meaning e.g. every Firefox user making use of the right to use a modified version built from source doesn’t have a license, thus being a “2nd class citizens” compared to users just using “official” licensed binaries. That’s completely incompatible with the ideas and ideals of open source.

Theora video can definitely be handled via plugin – that’s how Wikipedia is doing it (via Java).

@Alexander Limi: I can simply offer Flash, then it works for everyone including 300 million Firefox users. It saves me the need to build out and maintain an encoder farm and storage for Theora. IMHO that doesn’t do anything to encourage Theora adoption.

@Asa: From what I’ve seen of documentation on various chipsets most aren’t programmable by the implementer as @Olly Hodgson notes. Many seem non-programmable at all. H.264 chipsets are designed to be low cost and low power. Thanks to BlueRay and the other tech using H.264… these chips are already pretty cheap.

I suspect the QUALCOMM MSM7200 used in the G1 could be programmable, but it hasn’t exactly proven itself in power consumption either. In fact that’s generally considered to be the only major flaw in an otherwise fine phone.

Generally programmable chipsets like the Intel GMA 950 (graphics, but kinda similar) are used for more broad tasks at the expense of cost and power efficiency. Apple preferred the Intel GMA due to it’s low cost and that it was programmable allowing them to customize it for their graphics needs.

@Ian M: I was thinking something more native than Java… whose performance I think sucks. I still like what Wikipedia is doing though, but I don’t think the delay for loading Java is acceptable when the rest of video on the web is nearly instant playback.

The H.264 hardware is not really H.264 specific. It implements pieces of the decoder like DCT. These hardware assist modules are combined with software to make a H.264 decoder. Of course the documentation for all of the video decoding hardware is secret. You need access to this documentation and then you need to see if these hardware assist components can also be used in Theora. Theora is DCT based so I suspect the answer is yes.

The PowerVR people are the source of a lot of the H.264 decode hardware and they are totally closed source.

@Jon Smirl: You are correct though I believe there is a distinction between H.264 hardware acceleration and H.264 hardware decoding, the difference being the amount of work offset to hardware.

PowerVR I believe has a stronger hold on mobile than other fields, mainly thanks to PowerVR VXD and it’s use in some high profile phones.

Set top boxes and appliances can range from Broadcom to Conexant, STMicroelectronics, Sigma Designs, and TI. These are also important for Theora to take off for the same reasons that mobile is important. Eventually an Apple TV like box will take off. Theora being on that is important for all the same reasons.

I guess people don’t remember the Unisys snafu about a decade and a half ago. (For those of you too new to the net and/or computers to have a reason to recall this, Unisys owned patents related to the GIF file format. After effectively saying “here, use our image format” later “brains” to be decided to go after those using the file format. I remember a number of BBS and major online services getting letter.)

I don’t see why people don’t support open formats. You have to take into account that most Linux distros are bundling Firefox with their OS. Of course, they’re probably smart enough to get others, but there’s a large user base there that wants and expects an open-source solution. There’s no need to pay someone to support a format.

Unfortunately, I don’t think there will be enough of a push for an open format to derail h.264. Unless IE officially supports it, of course.

Leave a Reply

Your email address will not be published. Required fields are marked *