Apple refuses to implement Ogg Theora in Quicktime by default (as used by Safari), citing lack of hardware support and an uncertain patent landscape.

Google has implemented H.264 and Ogg Theora in Chrome, but cannot provide the H.264 codec license to third-party distributors of Chromium, and have indicated a belief that Ogg Theora’s quality-per-bit is not yet suitable for the volume handled by YouTube.

Opera refuses to implement H.264, citing the obscene cost of the relevant patent licenses.

Mozilla refuses to implement H.264, as they would not be able to obtain a license that covers their downstream distributors.

Microsoft has not commented on their intent to support

I think everyone agrees this is going nowhere and isn’t likely to change in the near future. For the sake of moving HTML5 forward, this is likely the best decision.

Here’s how I interpret everyone’s position:

Apple’s Argument

One of the undeniable perks behind H.264 right now is that there is hardware decoding available and used on on certain devices. One of the most notable is the iPhone. Using hardware decoding means your not using the CPU which results in better performance, and most importantly better battery life.

Thus far there’s no hardware Theora decoder on the market (if you know of any let me know, my research says none), which I suspect is why Apple is hesitant to jump on board. Until there’s hardware that’s proven to perform well, be cost-effective in the quantities Apple needs, and not be bombarded with patent infringement claims, I suspect they’d rather settle with H.264. The patent part is critical. Apple can update software to comply with patent wars pretty quickly, as many other companies have done with software in the past. Hardware is not so easy. Last minute hardware changes are harder to deal with than software because of the many things it impacts, and the inability to update at a later date.

I’m almost positive the lack of hardware support is the exact same reason Apple has been so against Flash support. Remember the YouTube application isn’t using VP6 like regular flash, it’s using H.264 (that’s why it took so long for all of YouTube to be available on the iPhone).

If there’s enough Theora content out there, there will likely be Theora decoder hardware made to meet market demand. To get to this point will be difficult with the amount of VP6 (Flash) and H.264 content already on the web. H.264 alone has a major head start in applications. VP6 has several years of video on the web now (and I still don’t think it has a hardware decoder on the market though that might be due to licensing again).

In the long run, I think mobile technology will improve enough to make this a somewhat unnecessary constraint. Mobile CPU’s and GPU’s are just starting to get to the caliber needed for video. Performance per watt should improve. Battery technology is just starting to get pushed to the limits. This is a good thing for Theora in the long run, but the question is how long?

Until it can be played with minimal impact on battery life, I don’t think any company who has a heavy investment in mobile will want to jump on board.

Google’s Argument

Google has money and can license H.264. Shocker. Google however has trouble when it comes to Chromium. I suspect Google doesn’t care too much about which way this goes since what they support in Chrome doesn’t mandate that YouTube support it. However if the encoding quality for a given bitrate is good enough, it becomes a viable option.

Regarding the quality argument, I’ll simply point to this comparison. I the quality today is comparable already, and likely to get better as the encoders improve. I’ll leave this discussion here.

Opera’s Argument

Opera says H.264 is to expensive to license. I don’t know what the costs are, and what they would be for Opera, but I’ll take their word on it. After all, the do have a product available for free download. While commercial and closed source, they don’t have Google’s revenue stream and I respect that.

Mozilla’s Argument

Mozilla can’t license for downstream Gecko use etc. I’m sure a good part of the argument is also that requiring licensing fees to use <video/> is bad for the web and open source. I agree.

Microsoft’s Argument

No comment. Historically they implemented <marquee/> but not the <blink/>. Make of that what you will.

<video/> could be supported by plugin if needed. I recall Adobe supporting SVG by plugin a few years ago.

Where to go from here?

I think there are a few possible outcomes. As for what I think are the most likely:

1. There’s a push for hardware decoding that makes Theora on mobile technically possible and working well. If Apple legally is satisfied and jumps on board that changes the game. As I stated earlier I think Google is mostly ambivalent since they support both right now. Opera doesn’t want H.264 anyway, so they are cool. IE 8 can likely be handled by a plugin. Apple really is the deciding factor. Theora is the future.
2. See what the web does. I suspect at least for a long while the web will just stick with Flash since it works on almost all desktops. For mobile the iPhone and Android make up pretty much the bulk of the mobile video market and that doesn’t look like it’s changing so fast. Content providers that want mobile will encode for mobile. That means 3 target platforms, not ideal but reasonable. H.264 and whatever Adobe adopts is the future.

I know how the media is interpreting all of this. How do other developers, and open source folks see it?

var httpRequest;
if (window.XMLHttpRequest) { // Mozilla, Safari, ...
    httpRequest = new XMLHttpRequest();
} else if (window.ActiveXObject) { // IE
    httpRequest = new ActiveXObject("Microsoft.XMLHTTP");
}

That’s right, IE 6 didn’t support the native xmlHttpRequest object (more here). This is just one of many examples in JavaScript and CSS. document.all anyone?

The end result of this problem became to be known as the “Web Standards” movement. Simply put it’s an idea that code should follow a standard that results in consistent output across all browsers on various platforms. Write once, run anywhere. While it’s taken years for this to manifest, it’s slowly become a reality. Firefox, Safari, Opera have fairly consistent rendering (at least in comparison to the mess of just a few years ago on the browser scene. IE 6 was fairly poor in terms of modern web development, but IE 7 made progress, and IE 8 is Microsoft’s greatest effort to date to bring their browser up to speed.

Unfortunately, all that “progress” is really catch up and making specs from several years ago work across browsers as the biggest area of problems (CSS 2) become a recommendation in 1998. 10 years and a few days ago. Web sites, and developers want to do new things.

As a result groups like WHATWG and HTML 5 formed with the intent to provide modern features such as 2D drawing (canvas), media playback, drag and drop, offline mode, and storage. All things desperately desired by web developers. Some of this stuff to date has been supplemented by the use of plugins such as Flash which can provide some of this functionality, in particular video. It should be noted the development of standards is a historically slow process since it’s essentially a bunch of geeks arguing until their keyboards wear out. This process takes a long time.

To speed things up, some once again bypassed standards and instead decided to implement things on their own:

• Apple implemented Canvas, which has some notable concerns regarding intellectual property, though can still be standardized (and is still in the HTML5 specs as I’m typing this).
• Google released Gears which originally provided offline support for web browsers, but has since expanded to allow for better desktop integration and running javascript in the background on browsers it supports.
• Yahoo just announced BrowserPlus, which is an API “that allows developers to create rich web applications with desktop capabilities” (that’s verbatim from their page).
• Mozilla added to Firefox 3 Online/Offline events, Web-based protocol handling, which are both part of the WHATWG Web Application 1.0 specs. In addition, APNG and text extensions to Canvas were added, though it’s noted that they are experimental and essentially Firefox only for now.

There are dozens of other little things in various browsers that are specific to them, but I won’t even go into that. These are just the more high profile things at this time.

Does anyone else see the problem here? At the rate things are progressing, it’s going to be pretty tough to build a competitive and user-friendly web application without requiring at least one framework being installed on the client side. This means before a user can enjoy a good web experience they will need to install:

• Standards Compliant Web Browser (may come with OS)
• Latest version of Adobe Flash (essentially the video standard right now)
• BrowserPlus (for some sites)
• Google Gears (for some sites)

This really is starting to feel like another browser-bubble, this one being more of a “browser feature bubble”. People rapidly trying to add features to meet application needs without standardizing. This time instead of building it all into the browser, they are also providing plugins to add the functionality. This is marginally better at best. While it allows for support on multiple browsers/platforms it restricts innovation since not all browsers/platforms are supported.

Google Gears still doesn’t support the iPhone, despite it running in Safari due to Apple’s restrictions on 3rd party software. It also doesn’t run on all portable Linux devices, or Blackberry’s. Not to mention Opera or Camino. It works on other Mozilla browsers that support extensions provided you tell your users to follow these instructions (you can essentially call that “unsupported”).

Yahoo’s BrowserPlus is no darling either. As of yet, it only supports Intel Mac’s running 10.4 or later (sorry PPC millions), and only Windows / Mac running Firefox 2+ or IE 7+. That leaves out millions of IE 6, Safari, and Opera users. Plus all of the mobile users (who are growing in number). Yahoo promises that will improve, but we’ve got to wait for Yahoo engineers to get around to that. If Yahoo is acquired by someone, who knows what that companies priorities will be for this project.

Apple implemented Canvas in Safari, Mozilla implemented it in Firefox. But it’s adoption has been slow from a developer’s point of view since IE doesn’t support it. There are several libraries (including one by Google) that attempt to add support, but they are pretty slow and don’t provide the same experience to IE users (in particular older IE 6 users). APNG is supported in Firefox 3 and Opera 9.5, both not released just yet. APNG degrades nicely, but if your using an animation, you want an animation for all users, not just Firefox and Opera users.

Yes, adoption can grow for canvas, Yahoo and Google can support more platforms and browsers, but that doesn’t fix the root problems here:

• The entire web is waiting on a company to update their enhancements to support a platform/browser. Open sourcing the code doesn’t really fix the problem since distribution is still problematic. Also just increases the chances of forking which makes it even more painful. Installing a browser in bits and pieces sucks. Users shouldn’t be subjected to that.
• There are multiple API’s to essentially do the same thing (drag & drop, offline support, video, etc.).

Does anyone else see this new form of fragmenting to be problematic?

While I agree the features each vendor is adding in are desired, I don’t think this is the right way to do it. One of a few things will happen here:

• After a period of time, one or more of these API’s will become deprecated as one becomes dominant. That’s a major pain for developers who choose the “wrong” one, and an expense for companies who are in that situation. Nobody wants to be in this situation.
• Things will continue to fragment and we’ll be back to the “Designed For Netscape” era, but it will be “Designed for Firefox 3+, Google Gears, Flash 9.0 r124, whatever”. I get a chill just thinking about it.
• Horrifically complicated code that essentially supports multiple products by using a subset of their functionality. This might be the “best” and it still sucks on a massive scale since it increases costs for development and decreases fun. You could use a JS library to abstract functionality to a more platform neutral API, but more libraries add overhead you may not want. Not to mention more code your relying on someone else for.

None of these are a good outcome.

I’m not saying that the standards route is perfect. There’s no binding requirement for anyone to adopt anything, or even accomplish a final spec. There are many half implemented specs, and many that never go anywhere. Perhaps that’s part of the problem. Perhaps it’s that standardization has historically been a closed process, though it’s slowly opening up (thanks WHATWG). Participation is also pretty touch as not many can manage to read all that email.

Browser vendors are not without their faults either. They are historically a slow to fully adopt specs, and generally don’t collaborate on where they will start their implementation. Partial support is as good as no support unless a it’s adopted uniformly across all popular browsers. Perhaps that’s a place that browser vendors can best help developers. By simply agreeing to implement XY.

That said, one could create a plugin that implements a standard. It’s already be done. Adobe released an SVG plugin (prior to the whole Macromedia deal as I’m not sure what the status is considering the Flash business is a closed source competitor), there’s also an XForms plugin for Mozilla browsers.

I really hope some caution is used before we have a browser feature bubble. This is just going to become a mess of technologies that nobody can keep up with. A lot of progress has been made in the past few years to fix mistakes of the past. Is this process of add-ons really the right method of giving web developers what they want?

To be fair, Google’s Aaron Boodman recently blogged regarding Google Gears and standards. And does indicate the desire to become become compatible with HTML5:

Currently, the Gears Database and LocalServer modules are not fully compatible with the HTML5 proposals for the same functionality. This is only because those specs were written after Gears was released, and not because of any desire to be different. In fact, we were involved in the design of both HTML5 specs, and we are currently implementing the proposal for database access.

This is good, but it’s still a moving target on a moving target as Google Gears is in beta and still being changed to match specs which are still evolving.

To avoid any misconceptions, I do think these features are a good thing. I like canvas, offline support, drag & drop integration, and want to implement them myself as well. But I don’t think the process to add them has been successful thus far. I think it’s just asking for trouble later on and will take a long time to repair. This isn’t about Firefox extensions, or javascript libraries like jQuery, YUI or Prototype. I love them and use them every day. This is about having non standard ways to do things that clearly need to be standardized (and are in the process of being standardized), and encouraging other developers to implement a method that’s tied to a particular piece of software.

Who is at fault here? Pretty much everyone. Standards bodies are to slow for a fast moving industry for starters. Secondly browser vendors fail to coordinate their progress on spec implementations. While nobody expects CSS 3 to be implemented overnight, of vendors would agree on milestones in which certain features would be implemented, that would greatly help implementation for web developers. Lastly plugin developers need to ensure what they are offering is widely available, free of any licensing problems that would impede implementation in other products, and are on a parallel track for standardization.

The web wouldn’t have succeeded if you needed to install a plugin for a <a href=""/>, or <img/>.

David Baron has a must read blog post. Since the bulk of these older sites are on intranets, why not just allow administrators to enable compatibility mode on intranets? Seems like a perfectly logical solution that doesn’t hold back the web, and allows them to achieve their goals.

Daniel Glazman has another idea that I think is very valid.

• Microsoft should freeze, and I really mean it, its current IE6/IE7 HTML 4 engines, and drop that META tag idea.

Ditto. That needs to be a very deep and reliable freeze though.