This number presumably is at least partially based on a prediction by Cisco who makes routers that carry that internet traffic. They even bought Flip (an unfortunate tale) simply to help drive video on the internet and presumably drive up the need for routers, their main business. They might have a bias.

The reality is video already is, and will continue to be the vast majority of Internet traffic. However that doesn’t mean people are spending as much time on it as the numbers will make you believe. Video per minute of online activity is just insanely high bandwidth and expensive. The percentage of bandwidth is meaningless, what matters is where people spend their time.

Video compresses very poorly in comparison to text and audio (interestingly, speech compresses fairly well these days). Video these days either streaming or progressive download is still a very bandwidth intensive activity.

This webpage is likely under 160 KB for you and loads in about 300 ms. You’ll spend statistically a few minutes reading this, perhaps slightly less if you just skim for the numbers. The takeaway is experiencing: this blog post is about 160 KB over a few minutes assuming you visit the site, and don’t use an RSS reader (go ahead RSS users, try it).

Now lets figure out what video is typically. Most web video these days is still H.264, and generally 640×480 (480p) or 1280×720 (720p). Most laptops today are pretty cheap and just can’t handle 1080p, certainly their displays aren’t good enough to show a measurable difference, not to mention the buffering time annoys people, so we’ll ignore it’s existence.

I want to make this a little realistic, so to figure out a bitrate, we’ll use the standard formula for what Adobe calls ‘ideal’ in the H.264 Primer, and what most seem to use these days:

[image width] x [image height] x [framerate] x [motion rank] x 0.07 = [desired bitrate]

For 1280×720 that means:

1280 x 720 x 24 x 2 x 0.07 = 3,096,576 bps = ~3000 kbps

And for 640×480:

640 x 480 x 24 x 2 x 0.07 = 1,032,192 bps = ~1000 kbps

This sounds about right. As of today, Wikipedia lists the bandwidth range for 720p at 2–2.9 Mbit/sec and for 480p at 0.8–1 Mbit/sec. So we know we’re on track.

That’s all the math we really need to do. Google can do the conversions for us:

For 720p:

3000 kbps = 1,318.35938 Megabytes / hour = 22 MB / min

For 480p:

1000 kbps = 439.453125 Megabytes / hour  = 7.3 MB / min

Going back to the example of this blog permalink, at 160 KB for a few minutes of reading vs. 7,475 KB for one minute of 480p video. That is why video is 90% of traffic.

Video growth is huge, but it’s not nearly as one sided as the stats will lead you to believe.

WebM Support

WebM support will be growing quickly as Firefox 4 rolls out (Firefox upgrade adoption is legendary). Chrome commands sizable market share and is pushing the Chrome OS platform. Opera is also supporting WebM.

Apple and Microsoft could join the party and bundle WebM support along with the other codecs they support at any time, though they are licensors for H.264 and wouldn’t benefit from WebM market penetration. Microsoft’s implementation does allow for VP8 support if a codec is installed. I’m not aware of anything for Safari and am rather certain nothing can be done for the iPhone without Apple intervening.

On the hardware side AMD, ARM, Nvidia are backing WebM. Broadcom announced support, as did Qualicomm and TI. These are major vendors for mobile chips. Intel is working on stuff too.

H.264 Trouble

H.264 is problematic and bad for the web for many reasons I’ve mentioned here before as well as great posts by roc and shaver. I’ll leave it at that rather than rehash.

There was buzz a while back about H.264 being “free” (quotes intentional), but it’s not really “free” if you read the fine print. As Peter Csathy of Sorenson Media notes:

But, you say, MPEG LA recently announced that it will no longer charge royalties for the use of H.264. Yes, it’s true – MPEG LA recently bowed to mounting pressure from, and press surrounding, WebM and announced something that kind of sounds that way. But, I caution you to read the not-too-fine print. H.264 is royalty-free only in one limited case – for Internet video that is delivered free to end users. Read again: for (1) Internet delivery that is (2) delivered free to end users. In the words of MPEG LA’s own press release, “Products and services other than [those] continue to be royalty-bearing.”

That’s hardly “free”. That’s just one potential use case that’s now royalty exempt. The reason they are doing that is presumably if they can get H.264 adoption high enough, all the other cases will be paying and therefore subsidizing this one case.

WebM is licensed a little different: Patent wise, it’s irrevocably royalty free. License is about as liberal as you can get.

There’s no proprietary html, css, or images (GIF was, now it’s dead) used across the web. Why should video be any different? The key to success and growth has always been an open platform that’s low cost and encourages innovation.

Implementing Today

For anyone who suggests that this further fragments the market, that’s not really true. Adobe Flash actually creates an excellent shim to help migrate away from Flash to <video/>. Allow me to explain:

Adobe will soon be supporting WebM through Flash. Adobe already support H.264 in Flash. For legacy browsers and those who won’t support WebM, you have the option of delivering a Flash experience just like most websites do today. There are websites doing this today via Flash and H.264. For modern browsers you can just use <video/>. Once your non-WebM market share drops low enough, you can get rid of the Flash experience. Soon enough you’ll be able to push WebM to your Flash users. The benefit of switching your Flash experience to WebM as a middle step would be one encoding for both delivery mechanisms vs. using H.264 and WebM in parallel. Of course if you’re supporting mobile you likely need H.264 for a bit longer but likely use a smaller resolution and different profile for mobile consumption.

No matter what there will be two delivery mechanisms for those looking to push video using HTML5 to users today. The only thing that changes is the lean towards standardizing on the actively developed WebM codec vs. H.264.

All new technology has speed bumps, that’s the cost of being on the bleeding edge. However this is a positive turn as things are now starting to line up. The most awesome thing is that the codec, HTML5 specs, and some of the most popular browsers in the world are open and inviting feedback and contributions to improve things.

I’ve only played with it for a few minutes on my Core i7 MacBook Pro, and things seem very speedy and my CPU didn’t see much of a spike. Hopefully enough videos will take advantage of hardware decoding that this will be a nice improvement.

I still believe WebM is the better future, but H.264 hardware decoding does make Flash less painful for the moment.

WebM is an open, royalty-free, media file format designed for the web.

WebM defines the file container structure, video and audio formats. WebM files consist of video streams compressed with the VP8 video codec and audio streams compressed with the Vorbis audio codec. The WebM file structure is based on the Matroska container.

Google describes the license as “BSD-style”. A very good move since it’s liberal enough to encourage widespread open and proprietary inclusion. GPL is to viral for some potential adopters.

Software Support

For the browser side, Chromium and Firefox Nightly builds support WebM starting today. Opera and Google Chrome to come shortly.

Google also created patches against FFmpeg for encode as well as decode and created DirectShow filters which are available for download. I suspect by way of libavcodec we’ll see support in lots of other products in the near future.

Microsoft will support VP8 in Internet Explorer 9 if you have the VP8 codec installed. Not quite “support”, but better than nothing.

Adobe is also supporting VP8 in Flash, which means content producers can eventually kill VP7 and VP6 encoding and use VP8 to reach most of their audience. This is very important as encoding videos into several formats is costly and time consuming (I know this very well).

Hardware Support

Google has already said they are working with video and silicon vendors to add VP8 hardware acceleration to their chipsets. I suspect newer phones in the near future will be supporting it. Especially if they run Android.

Content

Google is supporting WebM in the HTML5 test for YouTube which I mentioned a few months ago. I suspect we’ll see lots more support in the very near future.

Supporters

Even more telling of the potential than the above is the list of supporters which contains some big names who can put a lot of weight behind hardware/software/content support. AMD (who owns ATI), NVIDIA, Marvell (lots of mobile chipsets), Qualcomm (think mobile chipsets), TI, Broadcom, ARM on the hardware side alone is impressive. If the majority of them add hardware support to their upcoming offerings, that will be game changing. On the software side leaves 1.5 holdouts in the web video world: Apple (1) and Microsoft (0.5).

This is a game changer.

IE is supported through “Chrome Frame” which is essentially the Chrome browser in IE’s chrome. Your really just browsing the YouTube site with Chrome. Google could use this as a way to get people away from Flash and IE and onto Chrome one way or another.

I discussed the h.264 debate in more depth a few months ago.

You have to wonder why we don’t want anything proprietary slipping into HTML5, or want proprietary image formats (GIF turned us off to that) but exceptions are made for video.

Edit 1/23/2010: More on the topic:

• Robert O’Callahan’s Video, Freedom And Mozilla
• Mike Shaver’s HTML5 video and codecs

Edit 5/21/2010: Thoughts on WebM.

On2 is really an unknown to most people but their product has an amazing reach thanks to Adobe Flash. VP6 notably was included in Flash 8 and really brought about the age of Flash video (think YouTube). On2 also has VP7 which is considered a H.264 competitor. VP3 was released as open source and lives on as OGG Theora.

Of course by buying On2 Google will not need to pay any licensing for it’s VP7 technology, they can then bundle it into Chrome, Android and Google Chrome OS (finally giving Linux decent web video support). They could also open source it similar to these platforms in hopes that it will gain ubiquity.

This does however leave me wondering if this pending On2 deal had any bearing on the decision to leave HTML 5 <video/> codec ambiguous. It’s noteworthy since Google is very involved in the HTML 5 efforts. As I mentioned last month licensing is really key. If VP7 were open sourced and it’s licensing were compatible to meet Apple and Mozilla’s needs (which could lead to inclusion in Safari and Firefox respectively), OGG Theora is potentially dead overnight. Given Google’s strategy so far of making technology open source in efforts to encourage adoption, I wouldn’t rule this out, though it would likely take a while to evaluate everything and make sure they legally have that option. Timeline could also come into play here. The web isn’t necessarily going to wait for Google. These reviews can potentially take a long time. No guarantee others will incorporate it either, though it’s a pretty good deal should licensing work.

It’s also interesting that now Microsoft has Windows Media Player, Apple has QuickTime, and Google has On2′s codec bundle. It’s not exactly a “player”, but considering it’s usage it’s just as important.

It’s going to be very interesting to see how this plays out. One thing that seems relatively certain is that Google just made web video more interesting.

Apple refuses to implement Ogg Theora in Quicktime by default (as used by Safari), citing lack of hardware support and an uncertain patent landscape.

Google has implemented H.264 and Ogg Theora in Chrome, but cannot provide the H.264 codec license to third-party distributors of Chromium, and have indicated a belief that Ogg Theora’s quality-per-bit is not yet suitable for the volume handled by YouTube.

Opera refuses to implement H.264, citing the obscene cost of the relevant patent licenses.

Mozilla refuses to implement H.264, as they would not be able to obtain a license that covers their downstream distributors.

Microsoft has not commented on their intent to support

I think everyone agrees this is going nowhere and isn’t likely to change in the near future. For the sake of moving HTML5 forward, this is likely the best decision.

Here’s how I interpret everyone’s position:

Apple’s Argument

One of the undeniable perks behind H.264 right now is that there is hardware decoding available and used on on certain devices. One of the most notable is the iPhone. Using hardware decoding means your not using the CPU which results in better performance, and most importantly better battery life.

Thus far there’s no hardware Theora decoder on the market (if you know of any let me know, my research says none), which I suspect is why Apple is hesitant to jump on board. Until there’s hardware that’s proven to perform well, be cost-effective in the quantities Apple needs, and not be bombarded with patent infringement claims, I suspect they’d rather settle with H.264. The patent part is critical. Apple can update software to comply with patent wars pretty quickly, as many other companies have done with software in the past. Hardware is not so easy. Last minute hardware changes are harder to deal with than software because of the many things it impacts, and the inability to update at a later date.

I’m almost positive the lack of hardware support is the exact same reason Apple has been so against Flash support. Remember the YouTube application isn’t using VP6 like regular flash, it’s using H.264 (that’s why it took so long for all of YouTube to be available on the iPhone).

If there’s enough Theora content out there, there will likely be Theora decoder hardware made to meet market demand. To get to this point will be difficult with the amount of VP6 (Flash) and H.264 content already on the web. H.264 alone has a major head start in applications. VP6 has several years of video on the web now (and I still don’t think it has a hardware decoder on the market though that might be due to licensing again).

In the long run, I think mobile technology will improve enough to make this a somewhat unnecessary constraint. Mobile CPU’s and GPU’s are just starting to get to the caliber needed for video. Performance per watt should improve. Battery technology is just starting to get pushed to the limits. This is a good thing for Theora in the long run, but the question is how long?

Until it can be played with minimal impact on battery life, I don’t think any company who has a heavy investment in mobile will want to jump on board.

Google’s Argument

Google has money and can license H.264. Shocker. Google however has trouble when it comes to Chromium. I suspect Google doesn’t care too much about which way this goes since what they support in Chrome doesn’t mandate that YouTube support it. However if the encoding quality for a given bitrate is good enough, it becomes a viable option.

Regarding the quality argument, I’ll simply point to this comparison. I the quality today is comparable already, and likely to get better as the encoders improve. I’ll leave this discussion here.

Opera’s Argument

Opera says H.264 is to expensive to license. I don’t know what the costs are, and what they would be for Opera, but I’ll take their word on it. After all, the do have a product available for free download. While commercial and closed source, they don’t have Google’s revenue stream and I respect that.

Mozilla’s Argument

Mozilla can’t license for downstream Gecko use etc. I’m sure a good part of the argument is also that requiring licensing fees to use <video/> is bad for the web and open source. I agree.

Microsoft’s Argument

No comment. Historically they implemented <marquee/> but not the <blink/>. Make of that what you will.

<video/> could be supported by plugin if needed. I recall Adobe supporting SVG by plugin a few years ago.

Where to go from here?

I think there are a few possible outcomes. As for what I think are the most likely:

1. There’s a push for hardware decoding that makes Theora on mobile technically possible and working well. If Apple legally is satisfied and jumps on board that changes the game. As I stated earlier I think Google is mostly ambivalent since they support both right now. Opera doesn’t want H.264 anyway, so they are cool. IE 8 can likely be handled by a plugin. Apple really is the deciding factor. Theora is the future.
2. See what the web does. I suspect at least for a long while the web will just stick with Flash since it works on almost all desktops. For mobile the iPhone and Android make up pretty much the bulk of the mobile video market and that doesn’t look like it’s changing so fast. Content providers that want mobile will encode for mobile. That means 3 target platforms, not ideal but reasonable. H.264 and whatever Adobe adopts is the future.

I know how the media is interpreting all of this. How do other developers, and open source folks see it?

Big Buck Bunny, the new open movie made using Blender is out. It’s rather good, and impressive when you realize it’s made with open source products, meaning the only barrier to making one yourself (assuming you’ve got a rendering farm, or the patience to let your workstation churn out the pixels) is your skills. You can download it from the website (h.264 available) or watch on YouTube. I’d recommend the download so you can appreciate the HD quality. Some more screenshots can be found on Wikimedia Commons.

The first open movie was Elephants Dream back in 2007. Elephants dream used proprietary audio software. As far as I can tell, Big Buck Bunny didn’t.

Between the two I think I like Elephants Dream more. It was a little darker, but struck me as a little more entertaining. That’s my personal opinion though. It will be interesting to see what the next one is.

• ARM 926 CPU capable of running up to 312 MHz – This isn’t that much more than the existing iPhone which is said to be underclocked to preserve battery life. Don’t expect much change here.
• 5 Megapixel camera support – Capable, but don’t expect to see 5MP. I suspect it will be upgraded to 3MP and no higher to conserve costs.
• MPEG4 / H.263 hardware accelerator – Sigh, no H.264 support. That’s a bummer. Apple could still use hardware support via another chip.
• Support for video telephony, streaming, recording and playback – I wonder if Apple plans to utilize this. Video telephony could work over 3G networks (AT&T already did it with an LG CU500v). But it would require a potentially reworking the location of the camera on the phone.
• 3G upgradeable with WCDMA coprocessor – Very interesting since this could allow Apple to offer the iPhone on CDMA networks, though the largest (Verizon) is going to become LTE a varient of GSM. That’s the largest CDMA network in the US. Still CDMA support will be needed for Japan.

Walt Mossberg initially said this is going down in 60 days, but now he’s retracted that statement. I still think a June timeline sounds about right. WWDC 2008 is June 9-13, 2008. Sounds about right.

Apple doesn’t want the iPhone to get the reputation of having poor battery life. Apple is said to have avoided 3G thus far because of power consumption reasons, instead opting for a lower powered EDGE chip. When Apple moves to 3G later this year, they will want to at a minimum keep the same battery life. Having Flash on the iPhone will mean a likely drop in battery life. Something they don’t want.

I suspect in 12-18 months when H.264/AAC is a more common encoding scheme, I think we’ll see a Flash component for QuickTime that can take advantage of the hardware on the iPhone. Right now there’s too much vc6 stuff out there.

Apple doesn’t want anything released to drain battery life or it will be accused of misleading consumers about average battery life. Keeping the CPU idle will help keep that time up.

I suspect the SDK will have some limitations on CPU cycles an app built can consume before it’s throttled in some way. For the exact same purpose. That’s fine for most things, most users won’t notice, but for video, any slowdown or bottleneck becomes very visible.