…the single most braindamaged file format that I have ever seen in my nineteen year career”.

That bug was actually one of two times where I brushed with Mork, that time learning, and another time shortly afterwards where I learned first hand how impossible it really is to work with as part of a hack I was trying to build and later abandoned. Perhaps it was my experience at the time that just made it impossible, perhaps it really was Mork.

Coincidentally Mork is now completely dead. I suspect the last bits were actually Thunderbird who used Mork pretty extensively as Mail really is a giant database of messages, and another giant database of addresses that you interact with. I’m speculating as I haven’t been following that closely.

Long live SQLite. I still find you slightly odd at times, especially your locking which seems foreign to me, however I just think of Mork and all of a sudden, it’s OK.

The most amusing, interesting and strange comment header I’ve ever found in a script goes to mork.pl for this:

##############################################################################
#
# And Now, The Ugly Truth Laid Bare:
#
#   In Netscape Navigator 1.0 through 4.0, the history.db file was just a
#   Berkeley DBM file.  You could trivially bind to it from Perl, and
#   pull out the URLs and last-access time.  In Mozilla, this has been
#   replaced with a "Mork" database for which no tools exist.
#
#   Let me make it clear that McCusker is a complete barking lunatic.
#   This is just about the stupidest file format I've ever seen.
#
#       http://www.mozilla.org/mailnews/arch/mork/primer.txt
#       http://jwz.livejournal.com/312657.html
#       http://www.jwz.org/doc/mailsum.html
#       http://bugzilla.mozilla.org/show_bug.cgi?id=241438
#
#   In brief, let's count its sins:
#
#     - Two different numerical namespaces that overlap.
#
#     - It can't decide what kind of character-quoting syntax to use:
#       Backslash?  Hex encoding with dollar-sign?
#
#     - C++ line comments are allowed sometimes, but sometimes // is just
#       a pair of characters in a URL.
#
#     - It goes to all this serious compression effort (two different
#       string-interning hash tables) and then writes out Unicode strings
#       without using UTF-8: writes out the unpacked wchar_t characters!
#
#     - Worse, it hex-encodes each wchar_t with a 3-byte encoding,
#       meaning the file size will be 3x or 6x (depending on whether
#       whchar_t is 2 bytes or 4 bytes.)
#
#     - It masquerades as a "textual" file format when in fact it's just
#       another binary-blob file, except that it represents all its magic
#       numbers in ASCII.  It's not human-readable, it's not hand-editable,
#       so the only benefit there is to the fact that it uses short lines
#       and doesn't use binary characters is that it makes the file bigger.
#       Oh wait, my mistake, that isn't actually a benefit at all.
#
# Pure comedy.
#
##############################################################################

The title of this blog post is in no way a play on Mork & Mindy

This little bit of information from Richard Hipp, founder of SQLite, I found to be particularly interesting:

“…The consensus there seems to be that ‘public domain’ is valid and is a proper subset of ‘open source’–except in France and Germany where the concept of ‘public domain’ is not recognized…”

In my opinion, as long as the project stipulates that all contributions be released as public domain (defined as intellectual property not owned or controlled by anyone, and available for use by anyone for any purpose without restriction) for perpetuity, I think that in itself is an open source license. It’s also the cleanest and most easy to read.

Adobe Air vs. Mozilla Prism vs. Microsoft Silverlight

Google Gears vs. HTML5 Offline support

Android vs. iPhone SDK vs. Symbian

Ruby On Rails vs. PHP

Not every case will have a true “winner”. That’s not really a bad thing. Choice is good. In some cases they will merge to form one standard, such as what’s likely for offline web applications.

What is interesting is that SQLite really dominates right now. Adobe Air, Mozilla Prism, Google Gears, Android, iPhone SDK (likely through Core Data API), Symbian, Ruby On Rails (default DB in 2.0), PHP 5 (bundled but disabled in php.ini by default). It’s becoming harder and harder to ignore that SQL survived the transition from mainframe to server, and now is going from server to client.

No longer is the term “database” purely referring to an expensive RAID5 machine in a datacenter running Oracle, MySQL, DB2 or Microsoft SQL Server. It can now refer to someone’s web browser, or mobile phone.

This has really just begun to have an impact on things. The availability of good information storage, retrieval, and sorting means much less of these poorly concocted solutions and much better applications. Client side databases are the next AJAX.

Edit [2/27/2008 9:14 AM EST]: Added Symbian, since they also use SQLite. Thanks Chris.

I should note I do not own either device, but from what I’ve read, it doesn’t matter. Hopefully by 2nd or 3rd generation when I’m on the market Apple will have come to their senses.

The problem is that Apple hasn’t provided a good method for anyone to provide sync with the iPod. Even on older devices it wasn’t pretty, but it was workable. There is no real sync API outside of iSync, which is Mac only and not a public API. Instead what was done is mount the iPod file system and send it your data. While not awesome, this has been workable and provided many Thunderbird users with mobile data.

On the iPhone and iPod touch it’s not even possible to easily mount the filesystem. The best method I’ve heard of is hacking it so you can mount your phone using sshfs with something like FUSE. I know myself and some Linux and Mac users (horray for MacFUSE!) could manage, but I wouldn’t dare try to explain to someone how to do that. From what I’ve read the iPhone manages data using SQLite 3. Therefore it seems possible to use mozStorage to interface with it. Perhaps someone with more experience with mozStorage and db’s generated outside mozStorage would know better about any potential compatibility issues.

Calendar:
/var/root/Library/Calendar/Calendar.sqlitedb

Address BookL
/var/root/Library/AddressBook/AddressBook.sqlitedb
/var/root/Library/AddressBook/AddressBookImages.sqlitedb

Seems to me Apple could turn this into an API pretty easily by re-creating the old iPod style USB storage device mounting and give us access to these files.

In my mind the ideal implementation would be for Apple to just have a iTunes embed a mini web server locally accessible (obviously) and create a REST api to work with all data on the device. That would open up a whole new way of interacting with the device. Essentially you would interact with 127.0.0.1:[port#] and GET/POST your way through the interface. Perfect for Calendar, Address Book, Notes, or whatever else Apple comes up with.

To get a vCard for someone:

  GET /contact/vcard/?last_name=Jones HTTP/1.1
  Host: localhost
  User-Agent: mozPod/0.3

vcard can also be done as xml (ooh DOM parsing for iPhone!), so there’s lots of possibilities.

To add a contact:

  POST /contact/add/ HTTP/1.1
  Host: localhost
  User-Agent: mozPod/0.3
  Content-Length: 450
  Content-Type: application/x-www-form-urlencoded

  fname=John&lname=Doe&address=1%20Large%20Road...

Though it would likely make sense to go with a more vcard like naming structure.

For calendar iCalendar or vCalendar would make most sense:

 GET /calendar/ical/?start=1-1-1970&end=1-19-2038 HTTP/1.1
  Host: localhost
  User-Agent: mozPod/0.3

You get the idea. Dead simple access. You could then use something likecurl to manipulate the iPhone’s address book via your existing application. It would be only accessible locally, so it’s as secure as your system. It would actually make things more secure since it would be one less reason for tinkering on the actual device, and it would open up a whole new market of iPod/iPhone software to go with all those accessories that have helped boost Apple’s market share. SyncML is cool, but SyncML is also a beast and reminds me of SOAP, WSDL etc.