On Apple’s Lack Of Adaptive Image Support In WebKit

It’s becoming clear to me that despite Apple having a huge chunk of the mobile web, it still treats the web as a second class citizen on iOS and Mac OS X. My latest battle is adaptive images, in particular for use in High DPI devices (“Retina” on the Mac). High DPI displays are awesome. I own an iPad 3, and it’s one of the greatest displays I’ve ever looked at. What I don’t get is why Apple is making it so difficult to take advantage of as a developer.

Currently, there’s no easy way to switch image resolutions based on the display being used. The basis of that isn’t Apple’s fault. Nobody thought of the problem when HTML was first created. All of the methods have ugly tradeoffs. They are hacks. Even Apple.com doesn’t have a great solution. They were doing image replacement (easier to read version here). Apple does however have a solution called image-set which looks like it will be in iOS 6 and Mac OS X 10.8.

That’s several months later. The iPad 3 was announced March 7, which was only about 2 weeks after the initial proposal. Why wasn’t a solution for web authors included when the iPad 3 shipped? It seems silly that there’s no API to properly interface with one of the most touted features of the new device. Of course there’s a way to take advantage of that brilliant display if you build a native app.

You could argue that Apple didn’t want to rush implementing something proprietary without discussing it with the community at large, but Apple has said in the past:

tantek (Mozilla): I think if you’re working on open standards, you should propose your features before you implement them and discuss that here.
smfr (Apple): We can’t do that.
sylvaing (Microsoft): We can’t do that either.

So Apple is pretty candid about reserving the right to implement features without discussion, yet nothing happened. It’s not that such a discussion would have been a “leak”. The iPhone 4′s display would be adequate justification for the feature. In fact it’s mentioned in the first sentence of that proposal in www-style by Edward O’Connor. So not disclosing the new product doesn’t seem to be the reason either. Apple could have done this a year ago without anyone being any wiser about the iPad 3′s display.

I can’t be the only one who’s scratching his head over this. Why didn’t the iPad 3 ship with a browser capable of providing an efficient way to switch images? The cynic in me would say “to encourage native app development”, but then why bother now?

The upside is Apple products have high OS adoption rates. All those retina iPad 3′s will be running iOS 6 relatively quickly. If it were a popular Android device I’d be much more concerned because we’d be dealing with 2 years of devices on a stale OS with no support. This is why we need more competition in mobile. We need web solutions to be a priority, not an afterthought.

As far as I’m aware image-set is also prefixed, but that’s another rant.

Perception Of Performance

Google is pervasive about associating Chrome with being fast. It’s was their primary pitch when they first announced it. Back when Firefox went 1.0, it wasn’t so much about speed but “not sucking” as all the geeks liked to say. Given IE 6 was the competition, that was likely the best marketing on earth. Sure it was faster, but sucking fast wasn’t nearly as good as not sucking. Not sucking encompassed the missing features, broken rendering, crashing, constant parade of security problems. It summarized the product surprisingly well for not being an official slogan by any means.

Google now launched Chrome for iOS. On the desktop Chrome and Safari both use WebKit, Chrome applies it’s own touches to make things faster. Notably they have their own JS engine. Safari also has it’s own JS engine. This is the secret sauce of performance. In the iOS world however Apple being the totalitarian dictator decided that iOS will provide WebKit and JS. If your app has any web browser functionality it will utilize these API’s and not implement it’s own engine. Verbatim:

2.17 Apps that browse the web must use the iOS WebKit framework and WebKit Javascript

Google Chrome for iOS however is Google integration into a reskinned experience of Safari. It’s the same browser. Just a new UI bolted on with some Google features integrated in. It’s not a separate browser. It’s a UI.

That however doesn’t stop Google’s marketing machine (I’d argue Apple marketing’s top rival) from putting “fast” as the second word:

Browse fast with Chrome, now available on your iPhone, iPod touch and iPad. Sign in to sync your personalized Chrome experience from your computer, and bring it with you anywhere you go.

It goes on to clarify:

  • Search and navigate fast, directly from the same box. Choose from results that appear as you type.

So Google isn’t truly misleading. It’s just very strategic wording.

The truth of the matter however is that Google Chrome on iOS is substantially slower than Safari. Safari uses Nitro to accelerate JavaScript, which powers most of the complicated websites that will slow down a browser on any modern device. Apple however restricts Nitro to Safari, and doesn’t let third party apps like Google Chrome use it. This is still the case as of iOS 5, and I believe is the case in iOS 6, though I haven’t personally verified that.

How much slower is Google Chrome on iOS in comparison to Safari? Well Here’s a SunSpider test I did on my iPad 3:

Safari

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 1817.9ms +/- 0.2%
--------------------------------------------

3d: 214.7ms +/- 1.1%
cube: 72.3ms +/- 0.7%
morph: 57.9ms +/- 0.9%
raytrace: 84.5ms +/- 2.2%

access: 224.9ms +/- 0.6%
binary-trees: 44.4ms +/- 1.7%
fannkuch: 96.2ms +/- 0.6%
nbody: 56.0ms +/- 0.0%
nsieve: 28.3ms +/- 2.7%

bitops: 141.0ms +/- 0.4%
3bit-bits-in-byte: 23.4ms +/- 1.6%
bits-in-byte: 29.5ms +/- 1.3%
bitwise-and: 37.8ms +/- 1.5%
nsieve-bits: 50.3ms +/- 0.7%

controlflow: 15.7ms +/- 2.2%
recursive: 15.7ms +/- 2.2%

crypto: 123.3ms +/- 0.6%
aes: 70.5ms +/- 0.5%
md5: 29.4ms +/- 1.3%
sha1: 23.4ms +/- 1.6%

date: 274.4ms +/- 0.7%
format-tofte: 139.8ms +/- 1.1%
format-xparb: 134.6ms +/- 0.7%

math: 175.1ms +/- 0.3%
cordic: 61.5ms +/- 0.8%
partial-sums: 74.4ms +/- 0.7%
spectral-norm: 39.2ms +/- 0.8%

regexp: 70.8ms +/- 0.6%
dna: 70.8ms +/- 0.6%

string: 578.0ms +/- 0.5%
base64: 78.3ms +/- 1.9%
fasta: 68.1ms +/- 0.9%
tagcloud: 109.5ms +/- 1.2%
unpack-code: 207.5ms +/- 1.2%
validate-input: 114.6ms +/- 0.7%

Google Chrome

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 7221.0ms +/- 0.1%
--------------------------------------------

3d: 802.7ms +/- 0.2%
cube: 230.9ms +/- 0.6%
morph: 297.3ms +/- 0.5%
raytrace: 274.5ms +/- 0.1%

access: 1112.0ms +/- 0.2%
binary-trees: 98.4ms +/- 1.1%
fannkuch: 609.6ms +/- 0.2%
nbody: 247.9ms +/- 0.2%
nsieve: 156.1ms +/- 0.4%

bitops: 957.2ms +/- 0.2%
3bit-bits-in-byte: 210.4ms +/- 0.6%
bits-in-byte: 232.9ms +/- 0.2%
bitwise-and: 188.5ms +/- 0.4%
nsieve-bits: 325.4ms +/- 0.2%

controlflow: 129.5ms +/- 0.3%
recursive: 129.5ms +/- 0.3%

crypto: 493.3ms +/- 0.2%
aes: 214.3ms +/- 0.4%
md5: 140.2ms +/- 0.3%
sha1: 138.8ms +/- 0.5%

date: 381.1ms +/- 0.3%
format-tofte: 214.2ms +/- 0.2%
format-xparb: 166.9ms +/- 0.5%

math: 770.7ms +/- 0.2%
cordic: 316.6ms +/- 0.2%
partial-sums: 243.2ms +/- 0.3%
spectral-norm: 210.9ms +/- 0.4%

regexp: 1340.2ms +/- 0.2%
dna: 1340.2ms +/- 0.2%

string: 1234.3ms +/- 0.6%
base64: 175.7ms +/- 0.5%
fasta: 205.6ms +/- 0.2%
tagcloud: 284.0ms +/- 2.3%
unpack-code: 370.1ms +/- 0.9%
validate-input: 198.9ms +/- 0.6%

Quite a bit slower.

So really, if you’re using Chrome on iOS, it’s because you absolutely love the design and integration with Google’s services, and are willing to trade off considerable JavaScript performance for those perks.

That however doesn’t stop many people from thinking it’s fast. Just in the past few minutes I’m able to find these Tweets among the thousands streaming across the web. I won’t mention or link to them directly (you could find them however if you wanted):

“Chrome for iOS is FAST, takes the mobile browsing experience to a new level.”

“I like it! It’s fast and can sync with Chrome desktop, which I use all of the time.”

“Liking #chrome on #iOS very slick, fast and clean looking”

“using chrome on my iphone right now.. cant believe how fast it is”

“That chrome for iOS is freaking fast but so basic. No tweet button, no add-on. Man I kinda disappointed. I give ‘em 1 ‘fore the update”

“Chrome for iOS? Hell yes!! So fast! #chrome”

“Google Chrome for iOS is fast.”

“Holy hell Chrome is fast on the iPad.”

The most touted feature isn’t actually a feature. It’s technically not even there. The numbers and the technology insist that it’s not (they prove it’s actually slower). But that’s what everyone is ranting and raving about. You could argue Google’s UI is faster, but I’d be highly skeptical that Google’s found Cocoa tricks Apple engineers haven’t. Perhaps a UI transition or two makes you think it’s faster or more responsive, however even that I can’t find any evidence of.

All the hard work the Google engineers did squeezing their services into a compact simple to use UI are ignored in favor of this non-existent feature. And as a developer who can’t ignore such a thing, I will say they did a great job with their UI.

I present to you, the power of marketing!

On Prefixing And Monobrowser Culture

I’ll say right off the bat that Daniel Glazman is right, and I fully support his message. The failure to alter the course of the web now will lead to headaches. Truthfully it’s already a headache, it’s just going to get worse. The IE Days were the dark ages of web development. I don’t want to go back to that.

In an ideal world, CSS prefixing wouldn’t be necessary. Browser vendors would spec things out, agree on a standard and implement it. That however is too rational, so CSS prefixing is an unfortunate reality. It outright won’t happen by the admission of Microsoft and Apple (pointed out by bz):

tantek (Mozilla): I think if you’re working on open standards, you should propose
your features before you implement them and discuss that here.
smfr (Apple): We can’t do that.
sylvaing (Microsoft): We can’t do that either.

Of course you can question if there’s really a legitimate need to work on standards in private. I’m personally skeptical a css property will leak the next iPhone.

It’s also worth noting Apple and Microsoft are both OS vendors and (cynically speaking) have interests that are explicitly contrary to the internet being a universal platform. Fragmenting the web and making it a more difficult platform to develop on is potentially in their interest. Not to different from their stance on h.264 of whom they are both licensors and thus haven’t implemented WebM.

I’m starting to second guess the permanence of prefixes. I personally think once there’s a standard the first release of a browser 12 months after standardization drops support for the prefix. Yes, this will break a few websites that never update. However it’s almost always an easy fix. I’d venture 95%+ of the time it could be done safely via a regex. Truth is you’re talking about 18-24 months from initial implementation in practice anyway. Possibly longer. A website that is so stale it can’t manage to deal with this in 1.5-2 years is in pretty poor shape to begin with. LESS and Sass can also be a big help in automating this. W3C CSS Validator already errors on prefixes. The tools to deal with this are in place today.

I should note dropping is unlikely to happen and thus wishful thinking.

A large part of this issue is how many websites are built these days, especially “mobile sites” which are typically separate sites bolted onto an API or even the backend database of a website. Often built by 3rd party vendors getting things passable and out the door is key. As a result every shortcut in the book is taken, including the absolute minimum in testing and compatibility.

For what it’s worth, this blog has only one prefix in use, and it’s coded as:

-moz-border-radius: 3px;
-khtml-border-radius: 3px;
-webkit-border-radius: 3px;
border-radius: 3px;

Which catches everyone. That takes all of 30 seconds at most to do.

Protecting Photo Privacy Via Browsers

Browsers can do more to protect users from inadvertently violating their own privacy. The NY Times today had an article about a topic that has been discussed in various circles several times now. The existence of geotagging data in photos. Many cameras, in particular smart phones like the iPhone can tag photos with GPS data. This is pretty handy for various purposes including organizing photos at a later date, iPhoto for example does a pretty nice job of it. Most photo applications however don’t make this information very visible, as a result many users don’t even know it exists, others simply forget.

What the problem looks like

The data, embedded in a photo looks something like this:

GPSLatitude                    : 57.64911
GPSLongitude                   : 10.40744
GPSPosition                    : 57.64911 10.40744

Which I could map.

Proposal

I propose that browsers need to have a content policy for when users upload images that can better protect them from uploading information they may not even realize. Here’s what I’m imagining:

The first time a user attempts to upload a photo that has EXIF or XMP data containing location they are prompted if they want it stripped from the image they are uploading. The original file remains unharmed, just the uploaded version won’t have the data. They can also choose to have the browser remember their preference to prevent being prompted in the future. They can revise their choice in the preferences window later if they want. This isn’t to different from how popups are handled. I thnk that per-site policy might be too confusing and not warranted, but perhaps I’m wrong.

Warning users about hidden information they may be revealing is a worthwhile effort. It’s only a matter of time before someone uses a “contest” or some other form of social engineering to solicit pictures that may reveal location data for users. Evildoers always find creative ways to exploit people.

Caveat

There are a notable caveat to this approach. The most notable is that flash uploaders would bypass this security measure though individual uploaders could do it themselves, or Adobe could do it, but I don’t think that’s enough of a turnoff to this approach. The same caveat applied to “private browsing” in browsers.

Prior Work

As far as I know no browser actually implements a security feature like this yet. There are a few Firefox Add-ons like Exif Viewer and FxIF (both written in pure JavaScript) that look at EXIF data but nothing that intercepts uploads.

Who Can Do It First?

I’m curious who can do it first. By add-on (seems like it should be possible at least in Firefox), and dare I say include in a browser itself? If this were earlier in the year I would have added this to the Summer of Code ideas list. Instead I’m just throwing it into the wind until 2011 rolls around.

Steve Jobs: Thoughts On Flash

Apple today published a letter from Steve Jobs aptly title “Thoughts on Flash“. What’s interesting isn’t so much what he said, but what he alluded to. This letter is about Flash, but it’s also about the future if the iPhone platform strategy. It also alludes to the future importance of WebKit and the open web. Lets walk through this. From his points:

First, there’s “Open”.

Steve is right. Flash isn’t really “open”. The iPhone isn’t either by any means. In fact it’s the most restricted computing platform in the world as far as I know. What he did note is that the iPhone uses WebKit and by proxy the web is the most open platform on the planet. That’s very noteworthy.

Second, there’s the “full web”.

Flash video itself isn’t that great by todays standards. That’s why sites like YouTube are serving HD video in H.264 rather than VP6. H.264, VP8 and Theora are the future. If all 3 or just one will survive remains to be seen. Regardless any of them can be played outside of Flash. The dependency on Flash to build a player is going away more and more each day.

Regarding games, this is a silly point. Almost all Flash games need a keyboard or mouse to work. They would never work with a touch screen. Nor would they scale to fit the screen. They would need to be significantly reworked/rewritten.

This is yet more alluding to WebKit and HTML5 where there are solutions already in place.

Third, there’s reliability, security and performance.

It’s pretty hard to dispute the reliability of Flash. It’s by far the driving force behind things like out of process plugins (OOPP) in Firefox among other browsers. It’s also been subject to lots of security vulnerabilities.

Fourth, there’s battery life.

The WSJ quotes Adobe’s Shantanu Narayen as saying the claims of Flash being battery draining are “patently false” but if you look at a CPU monitor while browsing a page with Flash, you can see the load increase quite a bit. Blocking flash on your browser does speed things up and keep your system cooler. I’m very suspect that Adobe has solved this in cell phones when they don’t even seem to have it under control in Windows.

Fifth, there’s Touch.

I already mentioned that mouse/keyboard interfaces just don’t work on the iPhone. No need to rehash that.

Sixth, the most important reason.

That’s actually a vague header. The reason is that they don’t want a third-party sitting between the iPhone API’s and developers. If that happens, developers are limited to what that third-party decides to implement. At the very most developers on the Flash platform get whatever is supported on all Flash platform (greatest common denominator).

That leaves Apple in a stupid position. They could implement killer features in the iPhone and create amazing API’s to take advantage of the features. But if Adobe doesn’t see a way to support things across platforms, or just doesn’t see the cost/benefit of implementing that feature, developers can’t use it. That marginalizes the product for Apple as well as developers.

Conclusion

I found this very interesting that he closed it like this:

New open standards created in the mobile era, such as HTML5, will win on mobile devices (and PCs too). Perhaps Adobe should focus more on creating great HTML5 tools for the future, and less on criticizing Apple for leaving the past behind.

In February of 2007 Steve Jobs wrote another letter on DRM. It’s noteworthy because in January 2009 Apple launched the ability to buy non-DRM protected music. The letter was really a hint at where things were going. He’s repeating the PR strategy that he used then, make no mistake of it.

I have a feeling the day will come where the App Store is deprecated in favor of promoting HTML5 based Applications either directly off the web or packed similar to how Dashboard Widgets are done now on Mac OS X. The App Store will be around for quite some time, but it will eventually morph.

That is why WebKit is so important to Apple. They want to abstract their OS to the point where they can provide very high level hooks into features they want developers to be able to use. The current iPhone App SDK was a solution created by Apple as a way to let developers put applications on the iPhone as an afterthought. The moderation is so that they can keep their security record intact and could shut down a malicious app before trouble becomes rampant. That puts them in the position where they can either approve all content and be viewed as sleazy by more conservative folks, or they can let everything go and accept that reputation. They obviously made their decision. Developers and some geeks hate it, but 99% of the rest of the world doesn’t even know about the process. Nobody wants to know how sausage is made.

The App Store will likely morph to feature Dashboard Widget like applications (not to different from Palm’s WebOS). Apple will still be able to cash in via that distribution point since they can use DRM giving them the only way to actually sell a protected application. You can view them online via you’re browser.

That’s my prediction. The day will come when the iPhone SDK that we know today will be deprecated. WebKit and HTML5 aren’t there today, but the day will come when they will be the tier 1 development platform for the iPhone. Steve Jobs is just laying the groundwork today.

For desktops, other platforms and browsers it’s worth noting that there’s a lot to gain here.

Microsoft Joins W3C SVG Working Group

Microsoft is joining the W3C SVG Working Group. Presumably that means there’s some interest in SVG for IE or Silverlight or both. I wonder what led to the change of heart.

I pretty much wrote off any chance of SVG being mainstream in 2005 when Adobe bought Macromedia. Adobe was previously somewhat of a SVG pusher, but Macromedia obviously is the home of Flash. As expected the SVG love dried up. The gap that Adobe filled was adding support for SVG to IE. If IE supports it natively that’s a game changer.

Gecko already has decent support for SVG. WebKit has support for a while. Opera has support as well. Without analyzing in too much detail there should be a subset that’s usable across current browsers and hopefully IE by the time IE 9.0 ships.

I must admit given the choice I’m still more interested in Microsoft supporting <canvas/>, but no word on that as of yet. I’m still hopeful.

Hooray for web standards!

Adventures With document.documentElement.firstChild

Here’s an interesting DOM test-case I ran across inadvertently yesterday.

For the purpose of this post assume the following markup:

< !DOCTYPE html>
<html>
<!– i broke the dom –>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <title>Testcase</title>
</head>
<body>
<p>Something</p>
</body>
</html>

If I use document.documentElement.firstChild I don’t get consistent behavior. In Firefox and IE I get the <head/> element, which is what I was initially expecting. In WebKit (Safari/Chrome) and Opera. I get the HTML comment which I wasn’t.

Continue reading

Google Chrome OS

The big news over the past 24 hours is the announcement of Google Chrome OS. Effectively Google Chrome OS is a stripped down Linux Kernel with just enough to boot Chrome/WebKit as it’s main UI. The exact UI paradigm hasn’t been reveled as of yet. Google claims:

Speed, simplicity and security are the key aspects of Google Chrome OS. We’re designing the OS to be fast and lightweight, to start-up and get you onto the web in a few seconds. The user interface is minimal to stay out of your way, and most of the user experience takes place on the web. And as we did for the Google Chrome browser, we are going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with viruses, malware and security updates. It should just work.

It’s an interesting and somewhat bold statement.

Continue reading

Chromium’s WebKit Fork Is No More

This is very cool news for both WebKit sand Chromium. Chromium will no longer use a forked version of WebKit. This will mean more contributions directly to WebKit and a more current Chromium.

I wish all browser vendors could agree and sync engines a bit more so that Safari/Chrome would ship the same version of WebKit rather than stagger based on their own release schedule. Same for Mozilla/Flock etc. While very difficult in many respects it would make it much easier on web developers to have less products out there to test against. I think it’s unlikely to happen, but that doesn’t mean it wouldn’t be convenient. Some Web Developer bias speaking here.

[Via: Tony Chang]

Is IE8 Trident’s Last Stand?

Randall C. Kennedy at InfoWorld wrote:

IE8 is the last version of the Internet Explorer Web browser. At least, that’s what I’m hearing through the grapevine. It seems that Microsoft is preparing to throw in the towel on its Internet Explorer engine once and for all.

There were rumors earlier this year that the IE team was looking at WebKit a few months ago. I said then and I still think that’s a real perilous approach considering the legacy they need to somehow support. The other approach is to start over, something that’s possibly on the works.

Any truth to these claims? I don’t know. Though I’d be curious to see how Microsoft handles it’s customers who expect old applications to keep working and others who want Microsoft to catch up with progress. I doubt they can go either way 100%. Which way will they lean? I think that’s anyone’s guess.