I’ve only played with it for a few minutes on my Core i7 MacBook Pro, and things seem very speedy and my CPU didn’t see much of a spike. Hopefully enough videos will take advantage of hardware decoding that this will be a nice improvement.

I still believe WebM is the better future, but H.264 hardware decoding does make Flash less painful for the moment.

First, there’s “Open”.

Steve is right. Flash isn’t really “open”. The iPhone isn’t either by any means. In fact it’s the most restricted computing platform in the world as far as I know. What he did note is that the iPhone uses WebKit and by proxy the web is the most open platform on the planet. That’s very noteworthy.

Second, there’s the “full web”.

Flash video itself isn’t that great by todays standards. That’s why sites like YouTube are serving HD video in H.264 rather than VP6. H.264, VP8 and Theora are the future. If all 3 or just one will survive remains to be seen. Regardless any of them can be played outside of Flash. The dependency on Flash to build a player is going away more and more each day.

Regarding games, this is a silly point. Almost all Flash games need a keyboard or mouse to work. They would never work with a touch screen. Nor would they scale to fit the screen. They would need to be significantly reworked/rewritten.

This is yet more alluding to WebKit and HTML5 where there are solutions already in place.

Third, there’s reliability, security and performance.

It’s pretty hard to dispute the reliability of Flash. It’s by far the driving force behind things like out of process plugins (OOPP) in Firefox among other browsers. It’s also been subject to lots of security vulnerabilities.

Fourth, there’s battery life.

The WSJ quotes Adobe’s Shantanu Narayen as saying the claims of Flash being battery draining are “patently false” but if you look at a CPU monitor while browsing a page with Flash, you can see the load increase quite a bit. Blocking flash on your browser does speed things up and keep your system cooler. I’m very suspect that Adobe has solved this in cell phones when they don’t even seem to have it under control in Windows.

Fifth, there’s Touch.

I already mentioned that mouse/keyboard interfaces just don’t work on the iPhone. No need to rehash that.

Sixth, the most important reason.

That’s actually a vague header. The reason is that they don’t want a third-party sitting between the iPhone API’s and developers. If that happens, developers are limited to what that third-party decides to implement. At the very most developers on the Flash platform get whatever is supported on all Flash platform (greatest common denominator).

That leaves Apple in a stupid position. They could implement killer features in the iPhone and create amazing API’s to take advantage of the features. But if Adobe doesn’t see a way to support things across platforms, or just doesn’t see the cost/benefit of implementing that feature, developers can’t use it. That marginalizes the product for Apple as well as developers.

Conclusion

I found this very interesting that he closed it like this:

New open standards created in the mobile era, such as HTML5, will win on mobile devices (and PCs too). Perhaps Adobe should focus more on creating great HTML5 tools for the future, and less on criticizing Apple for leaving the past behind.

In February of 2007 Steve Jobs wrote another letter on DRM. It’s noteworthy because in January 2009 Apple launched the ability to buy non-DRM protected music. The letter was really a hint at where things were going. He’s repeating the PR strategy that he used then, make no mistake of it.

I have a feeling the day will come where the App Store is deprecated in favor of promoting HTML5 based Applications either directly off the web or packed similar to how Dashboard Widgets are done now on Mac OS X. The App Store will be around for quite some time, but it will eventually morph.

That is why WebKit is so important to Apple. They want to abstract their OS to the point where they can provide very high level hooks into features they want developers to be able to use. The current iPhone App SDK was a solution created by Apple as a way to let developers put applications on the iPhone as an afterthought. The moderation is so that they can keep their security record intact and could shut down a malicious app before trouble becomes rampant. That puts them in the position where they can either approve all content and be viewed as sleazy by more conservative folks, or they can let everything go and accept that reputation. They obviously made their decision. Developers and some geeks hate it, but 99% of the rest of the world doesn’t even know about the process. Nobody wants to know how sausage is made.

The App Store will likely morph to feature Dashboard Widget like applications (not to different from Palm’s WebOS). Apple will still be able to cash in via that distribution point since they can use DRM giving them the only way to actually sell a protected application. You can view them online via you’re browser.

That’s my prediction. The day will come when the iPhone SDK that we know today will be deprecated. WebKit and HTML5 aren’t there today, but the day will come when they will be the tier 1 development platform for the iPhone. Steve Jobs is just laying the groundwork today.

For desktops, other platforms and browsers it’s worth noting that there’s a lot to gain here.

What’s likely the biggest feature in the upcoming Photoshop release is Content-Aware Fill. I’m sure the photos used for the demo work exceptionally well, and better than in practice, but regardless it’s amazing technology and leap years ahead of what’s currently out there.

He is also right that third-party plug-ins do cause architectural issues for browser vendors. As of 10/2009 plug-ins accounted for at least 30% of Firefox crashes, a motivating factor for the new plug-in checker.

I will however object to a sentence:

Why? At the core, because Flash is the only de facto web standard based on a proprietary technology. There are numerous proprietary web content plugins — including Apple’s QuickTime — but Flash is the only one that’s so ubiquitous that it’s a de facto standard. Flash is the way video is delivered over the web, and Adobe completely controls Flash. No other aspect of the web works like this. HTML, CSS, and JavaScript are all open standards, with numerous implementations, including several that are open source.

…

Apple isn’t trying to replace Flash with its own proprietary thing. They’re replacing it with H.264 and HTML5. This is good for everyone but Adobe.

I included an earlier paragraph since I think the context is important. H.264 is not like HTML, CSS, and JavaScript. It’s patent-encumbered much like GIF was. Your trading Adobe for MPEG LA. The difference between H.264 and Flash is browser/OS vendors can control the implementation. It’s still proprietary technology.

I should note that I’m not a fan of Flash either, as a result there’s none on this blog. Even videos I link to are static images for performance and aesthetic reasons.

I pretty much wrote off any chance of SVG being mainstream in 2005 when Adobe bought Macromedia. Adobe was previously somewhat of a SVG pusher, but Macromedia obviously is the home of Flash. As expected the SVG love dried up. The gap that Adobe filled was adding support for SVG to IE. If IE supports it natively that’s a game changer.

Gecko already has decent support for SVG. WebKit has support for a while. Opera has support as well. Without analyzing in too much detail there should be a subset that’s usable across current browsers and hopefully IE by the time IE 9.0 ships.

I must admit given the choice I’m still more interested in Microsoft supporting <canvas/>, but no word on that as of yet. I’m still hopeful.

Hooray for web standards!

var httpRequest;
if (window.XMLHttpRequest) { // Mozilla, Safari, ...
    httpRequest = new XMLHttpRequest();
} else if (window.ActiveXObject) { // IE
    httpRequest = new ActiveXObject("Microsoft.XMLHTTP");
}

That’s right, IE 6 didn’t support the native xmlHttpRequest object (more here). This is just one of many examples in JavaScript and CSS. document.all anyone?

The end result of this problem became to be known as the “Web Standards” movement. Simply put it’s an idea that code should follow a standard that results in consistent output across all browsers on various platforms. Write once, run anywhere. While it’s taken years for this to manifest, it’s slowly become a reality. Firefox, Safari, Opera have fairly consistent rendering (at least in comparison to the mess of just a few years ago on the browser scene. IE 6 was fairly poor in terms of modern web development, but IE 7 made progress, and IE 8 is Microsoft’s greatest effort to date to bring their browser up to speed.

Unfortunately, all that “progress” is really catch up and making specs from several years ago work across browsers as the biggest area of problems (CSS 2) become a recommendation in 1998. 10 years and a few days ago. Web sites, and developers want to do new things.

As a result groups like WHATWG and HTML 5 formed with the intent to provide modern features such as 2D drawing (canvas), media playback, drag and drop, offline mode, and storage. All things desperately desired by web developers. Some of this stuff to date has been supplemented by the use of plugins such as Flash which can provide some of this functionality, in particular video. It should be noted the development of standards is a historically slow process since it’s essentially a bunch of geeks arguing until their keyboards wear out. This process takes a long time.

To speed things up, some once again bypassed standards and instead decided to implement things on their own:

• Apple implemented Canvas, which has some notable concerns regarding intellectual property, though can still be standardized (and is still in the HTML5 specs as I’m typing this).
• Google released Gears which originally provided offline support for web browsers, but has since expanded to allow for better desktop integration and running javascript in the background on browsers it supports.
• Yahoo just announced BrowserPlus, which is an API “that allows developers to create rich web applications with desktop capabilities” (that’s verbatim from their page).
• Mozilla added to Firefox 3 Online/Offline events, Web-based protocol handling, which are both part of the WHATWG Web Application 1.0 specs. In addition, APNG and text extensions to Canvas were added, though it’s noted that they are experimental and essentially Firefox only for now.

There are dozens of other little things in various browsers that are specific to them, but I won’t even go into that. These are just the more high profile things at this time.

Does anyone else see the problem here? At the rate things are progressing, it’s going to be pretty tough to build a competitive and user-friendly web application without requiring at least one framework being installed on the client side. This means before a user can enjoy a good web experience they will need to install:

• Standards Compliant Web Browser (may come with OS)
• Latest version of Adobe Flash (essentially the video standard right now)
• BrowserPlus (for some sites)
• Google Gears (for some sites)

This really is starting to feel like another browser-bubble, this one being more of a “browser feature bubble”. People rapidly trying to add features to meet application needs without standardizing. This time instead of building it all into the browser, they are also providing plugins to add the functionality. This is marginally better at best. While it allows for support on multiple browsers/platforms it restricts innovation since not all browsers/platforms are supported.

Google Gears still doesn’t support the iPhone, despite it running in Safari due to Apple’s restrictions on 3rd party software. It also doesn’t run on all portable Linux devices, or Blackberry’s. Not to mention Opera or Camino. It works on other Mozilla browsers that support extensions provided you tell your users to follow these instructions (you can essentially call that “unsupported”).

Yahoo’s BrowserPlus is no darling either. As of yet, it only supports Intel Mac’s running 10.4 or later (sorry PPC millions), and only Windows / Mac running Firefox 2+ or IE 7+. That leaves out millions of IE 6, Safari, and Opera users. Plus all of the mobile users (who are growing in number). Yahoo promises that will improve, but we’ve got to wait for Yahoo engineers to get around to that. If Yahoo is acquired by someone, who knows what that companies priorities will be for this project.

Apple implemented Canvas in Safari, Mozilla implemented it in Firefox. But it’s adoption has been slow from a developer’s point of view since IE doesn’t support it. There are several libraries (including one by Google) that attempt to add support, but they are pretty slow and don’t provide the same experience to IE users (in particular older IE 6 users). APNG is supported in Firefox 3 and Opera 9.5, both not released just yet. APNG degrades nicely, but if your using an animation, you want an animation for all users, not just Firefox and Opera users.

Yes, adoption can grow for canvas, Yahoo and Google can support more platforms and browsers, but that doesn’t fix the root problems here:

• The entire web is waiting on a company to update their enhancements to support a platform/browser. Open sourcing the code doesn’t really fix the problem since distribution is still problematic. Also just increases the chances of forking which makes it even more painful. Installing a browser in bits and pieces sucks. Users shouldn’t be subjected to that.
• There are multiple API’s to essentially do the same thing (drag & drop, offline support, video, etc.).

Does anyone else see this new form of fragmenting to be problematic?

While I agree the features each vendor is adding in are desired, I don’t think this is the right way to do it. One of a few things will happen here:

• After a period of time, one or more of these API’s will become deprecated as one becomes dominant. That’s a major pain for developers who choose the “wrong” one, and an expense for companies who are in that situation. Nobody wants to be in this situation.
• Things will continue to fragment and we’ll be back to the “Designed For Netscape” era, but it will be “Designed for Firefox 3+, Google Gears, Flash 9.0 r124, whatever”. I get a chill just thinking about it.
• Horrifically complicated code that essentially supports multiple products by using a subset of their functionality. This might be the “best” and it still sucks on a massive scale since it increases costs for development and decreases fun. You could use a JS library to abstract functionality to a more platform neutral API, but more libraries add overhead you may not want. Not to mention more code your relying on someone else for.

None of these are a good outcome.

I’m not saying that the standards route is perfect. There’s no binding requirement for anyone to adopt anything, or even accomplish a final spec. There are many half implemented specs, and many that never go anywhere. Perhaps that’s part of the problem. Perhaps it’s that standardization has historically been a closed process, though it’s slowly opening up (thanks WHATWG). Participation is also pretty touch as not many can manage to read all that email.

Browser vendors are not without their faults either. They are historically a slow to fully adopt specs, and generally don’t collaborate on where they will start their implementation. Partial support is as good as no support unless a it’s adopted uniformly across all popular browsers. Perhaps that’s a place that browser vendors can best help developers. By simply agreeing to implement XY.

That said, one could create a plugin that implements a standard. It’s already be done. Adobe released an SVG plugin (prior to the whole Macromedia deal as I’m not sure what the status is considering the Flash business is a closed source competitor), there’s also an XForms plugin for Mozilla browsers.

I really hope some caution is used before we have a browser feature bubble. This is just going to become a mess of technologies that nobody can keep up with. A lot of progress has been made in the past few years to fix mistakes of the past. Is this process of add-ons really the right method of giving web developers what they want?

To be fair, Google’s Aaron Boodman recently blogged regarding Google Gears and standards. And does indicate the desire to become become compatible with HTML5:

Currently, the Gears Database and LocalServer modules are not fully compatible with the HTML5 proposals for the same functionality. This is only because those specs were written after Gears was released, and not because of any desire to be different. In fact, we were involved in the design of both HTML5 specs, and we are currently implementing the proposal for database access.

This is good, but it’s still a moving target on a moving target as Google Gears is in beta and still being changed to match specs which are still evolving.

To avoid any misconceptions, I do think these features are a good thing. I like canvas, offline support, drag & drop integration, and want to implement them myself as well. But I don’t think the process to add them has been successful thus far. I think it’s just asking for trouble later on and will take a long time to repair. This isn’t about Firefox extensions, or javascript libraries like jQuery, YUI or Prototype. I love them and use them every day. This is about having non standard ways to do things that clearly need to be standardized (and are in the process of being standardized), and encouraging other developers to implement a method that’s tied to a particular piece of software.

Who is at fault here? Pretty much everyone. Standards bodies are to slow for a fast moving industry for starters. Secondly browser vendors fail to coordinate their progress on spec implementations. While nobody expects CSS 3 to be implemented overnight, of vendors would agree on milestones in which certain features would be implemented, that would greatly help implementation for web developers. Lastly plugin developers need to ensure what they are offering is widely available, free of any licensing problems that would impede implementation in other products, and are on a parallel track for standardization.

The web wouldn’t have succeeded if you needed to install a plugin for a <a href=""/>, or <img/>.

Apple doesn’t want the iPhone to get the reputation of having poor battery life. Apple is said to have avoided 3G thus far because of power consumption reasons, instead opting for a lower powered EDGE chip. When Apple moves to 3G later this year, they will want to at a minimum keep the same battery life. Having Flash on the iPhone will mean a likely drop in battery life. Something they don’t want.

I suspect in 12-18 months when H.264/AAC is a more common encoding scheme, I think we’ll see a Flash component for QuickTime that can take advantage of the hardware on the iPhone. Right now there’s too much vc6 stuff out there.

Apple doesn’t want anything released to drain battery life or it will be accused of misleading consumers about average battery life. Keeping the CPU idle will help keep that time up.

I suspect the SDK will have some limitations on CPU cycles an app built can consume before it’s throttled in some way. For the exact same purpose. That’s fine for most things, most users won’t notice, but for video, any slowdown or bottleneck becomes very visible.

Adobe Air vs. Mozilla Prism vs. Microsoft Silverlight

Google Gears vs. HTML5 Offline support

Android vs. iPhone SDK vs. Symbian

Ruby On Rails vs. PHP

Not every case will have a true “winner”. That’s not really a bad thing. Choice is good. In some cases they will merge to form one standard, such as what’s likely for offline web applications.

What is interesting is that SQLite really dominates right now. Adobe Air, Mozilla Prism, Google Gears, Android, iPhone SDK (likely through Core Data API), Symbian, Ruby On Rails (default DB in 2.0), PHP 5 (bundled but disabled in php.ini by default). It’s becoming harder and harder to ignore that SQL survived the transition from mainframe to server, and now is going from server to client.

No longer is the term “database” purely referring to an expensive RAID5 machine in a datacenter running Oracle, MySQL, DB2 or Microsoft SQL Server. It can now refer to someone’s web browser, or mobile phone.

This has really just begun to have an impact on things. The availability of good information storage, retrieval, and sorting means much less of these poorly concocted solutions and much better applications. Client side databases are the next AJAX.

Edit [2/27/2008 9:14 AM EST]: Added Symbian, since they also use SQLite. Thanks Chris.

That said, if there is an implementation, I suspect it will be a special mobile version, and very MPEG-4 centric. By that I mean H.264 as the encouraged (if not only) video encoding, and AAC as the preferred audio format, with MP3.

There’s a simple reason for this. AAC, MP3, and H.264 can be processed using hardware decoding. This means the CPU isn’t needed, resulting in lower power consumption. Many mobile devices have specific hardware for this reason. There is unknown hardware in the iPhone, which may very well be for hardware decoding.

By leveraging hardware decoding it allows Apple to offer things like video without sacrificing thermals or battery life. Since Flash can now use H.264 as well, it could offload some of that complicated processing. The CPU itself contains PowerVR MBX 3D graphics.

This could allow for most Flash to work, with much lower power consumption. The downside to this is that VP6 encoded video wouldn’t be able to use hardware decoding. For many online video sites (which use VP6 since H.264 is still very new) you’d have to run off of the CPU meaning more thermals and power consumption. A notable exception is YouTube, which thanks to Google’s work with the Apple apparently uses H.264 by means of a custom application.

• Daniel Glazman mentions that there’s likely only room for 1 html editor. Go Live or DreamWeaver. I’m personally going to suggests DreamWeaver survives. Simply because it’s more robust. It’s advanced features such as editing code, php, etc. are much better than Go Live. It reaches a larger market. As for the impact on NVU? Well I guess that remains to be seen. I guess people will now be looking to see if DreamWeaver will be the only real commercial game in town.
• SVG. This I think is the largest impact this deal is going to have. The fate of SVG. Adobe has been pushing SVG since very early on. But now with Flash in their hands. How do they feel about an open standard? Will they perhaps decide to open Flash to relax critics and just push their software as the ‘ultimate Flash IDE’? Will they stifle the growth of SVG? Or will they perhaps make the Flash plugin render SVG just like QuickTime or other media plugins support multiple formats?
• Adobe has made a business of being cross platform (similar to Netscape). Their Acrobat Reader is available for virtually anything (even the Palm OS). Does this mean better Flash support for non-windows computers (which has historically sucked)?
• PDF + Flash + Shockwave = ?
• New products? Will Illustrator and Flash converge?
• How does Photoshop fit in? Will it integrate with Macromedia Products?

Overall this is a groundbreaking event. It’s not unexpected as their products were complementary for ages and it was inevitable for them to come together. It’s finally happened.