DBSlayer is a project by NYTimes a few years ago that seems to be somewhat forgotten but is pretty cool. It’s another MySQL proxy, but with a slight twist. Rather than use a binary protocol, or XML, they went with JSON. It supports things like connection pooling, round-robin distribution to slaves, automatic fallover, and it’s mutithreaded. It’s pretty fast and easy to work with. It’s almost like turning a MySQL database into a REST API. You pass a SQL query as a query argument and it gives you a JSON response.

Once you start it you can do something query using a request like:

http://localhost:9090/db?%7B%22SQL%22%3A%22SELECT%20*%20FROM%20facts%20WHERE%20id%3D1%3B%22%7D%20

That will give you a JSON object containing the result of your query.

So doing that in node.js is roughly:

var sql = '{"SQL":"SELECT * FROM facts WHERE id=7;"}';

var http = require('http');
var client = http.createClient(9090, 'localhost');

var request = client.request('GET', '/db?' + escape(sql), {});
request.end();
request.on('response', function (response) {
  console.log('STATUS: ' + response.statusCode);
  console.log('HEADERS: ' + JSON.stringify(response.headers));
  response.setEncoding('utf8');
  response.on('data', function (chunk) {
    console.log('BODY: ' + chunk);
  });
});

Running that looks like this:

$ node test.js
STATUS: 200
HEADERS: {"date":"Fri, 27 May 2011 02:02:27 GMT","server":"dbslayer/beta-12","content-type":"text/plain; charset=utf-8","content-length":"290","connection":"close"}
BODY: {"RESULT" : {"HEADER" : ["id" , "fact" , "author" , "ip" , "timestamp"] , "ROWS" : [[7 , "1+1=2" , "raccettura" , "127.0.0.1" , 123456]] , "TYPES" : ["MYSQL_TYPE_LONG" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_LONGLONG"]} , "SERVER" : "db"}

You could obviously clean that up and create a little library to hide the HTTP parts.

It’s an interesting JS centric way to abstract your database while maintaining SQL level control. JSON is becoming the new XML.

Firefox 3.5 is around the corner. For those who don’t pay attention to development here’s the big features worth checking out. There are lots more, but these are my favorites:

User Centric Features

Private Browsing – Officially it’s called “Private Browsing” but most know it as “porn mode”. Simply put once you turn on the feature nothing about your browsing is saved to your computer until you turn it off. No browser history, cookies, cache, no passwords, download list. Great for shared computers where you may not want the next person to know where you shopped, what you bought etc.

Faster JavaScript – Everyone is doing it. Firefox 3.5 now ships with TraceMonkey which uses a technique, called trace trees to add just-in-time native code compilation to SpiderMonkey, the JS engine in Firefox. Bottom line: faster JavaScript makes JavaScript powered sites like Gmail way faster.

Faster Awesomebar – The awesomebar is a fast way of browsing the web, but the UI can sometimes get a little sluggish. Some awesome work has been done to optimize it for better performance. Faster UI = better browsing experience.

Better Awesomebar – The Awesomebar got a few enhancements including autocomplete for tagging, which is extremely handy as well as editing tags on multiple bookmarks at the same time.

Undo Closed Window – We’ve all done it before. Now you can undo a closed window just like a closed tab.

Drag Tab To New Window – Previously you could drag/drop to reorder tabs. Now you can drag a tab off the tab bar to move it into its own window. This may sound trivial but it actually makes organizing tabs much easier.

Video/Audio – Firefox 3.5 supports the new HTML5 <video/> and <audio/> tags. Specifically it supports Vorbis in Ogg containers, as well as WAV with support for more formats expected in the future. I’ve discussed open video before and suggest learning more about how important this is there.

SSL Error Pages Suck Less – The error pages shown when there is an SSL error were pretty tough on users since they didn’t display anything helpful. The new error pages are a bit more helpful. The bug implementing the changes has tons of details on the changes.

Geolocation – Simply put a website can (if you allow it) gather information about your internet connection and using a location service (provided by Google by default) will calculate your location. No more needing to constantly type in your zip code, or city name to get local information. For privacy you need to explicitly allow it. Geolocation is in a word awesome.

New Icon – Well, it’s not really new. It’s “refreshed” I guess. It’s not a huge change, but it does look really sharp, especially in more modern operating systems that use larger icons like Mac OS X. Alex Faaborg has it on his blog in various sizes for you see.

Developer Centric Features

Web Workers – My personal favorite is web workers. Essentially its background processing in a separate thread for JavaScript. No more locking up the browser’s UI because you need to do some complicated JS calculations. I’ll leave it to the documentation linked above for examples. Very handy stuff.

@font-face – Designers have long been frustrated with the lack of font options on the web. They often resort to using images and flash as a way to expand their font options. With @font-face it’s now possible to use custom fonts and reference them via css. There is however the issue of licensing of fonts used on a webpage since the font file itself is accessible via a web browser.

Native JSON Enough said. Native JSON is fast. var obj = JSON.parse(someJS);

Cross Site xhr – xmlHttpRequest() has ushered in a new era of JavaScript. It’s not however without some serious limitations. One of the most obvious limitations is that you can’t use it across hostnames. Until now.

There’s more cool toys, but these are my favorite.

Still not convinced of all the new stuff? Check out this demo, then look at the source behind it. It’s pretty impressive.

Gripes:

• I’m not sure I agree with the authors decision to use JSON as the format. Considering this will be used mostly (if not only) to keep tabs on xml documents (RSS, ATOM mostly) it seems more correct to use XML. Presumably the reason for JSON is that it’s computationally easier to parse. The NYTimes created a database abstraction layer called DBSlayer that uses JSON rather than a binary protocol to avoid the need for a client. The advantage of JSON over xml is JSON is easy to parse, and I agree. JSON also tends to be lighter since you don’t have opening and closing tags surrounding all your data. Still, why introduce JSON to XML world?
• SUP needs index files. Google made a great move with it’s Sitemaps Protocol by allowing for Sitemap Indexes. Essentially it’s a bootstrap that lists several index files. Google further restricted that a Sitemap shall contain no more than 50,000 items and be no larger than 10MB. For a very popular site such as Facebook the SUP feed could become painful to parse as 1 file. Having a SUP Index would be much better. I’d essentially copy Google’s design and rules regarding size, I think they work rather well.
• SUP should allow for both using SUP-IDs or RSS URL’s. SUP could be more useful if it were an index of all RSS feeds in some cases. Having that option in the protocol would make sense and future proof. I’m sure Google wouldn’t mind it.

Likes:

• Unlike using XMPP which can be tough for a startup to implement (look at Twitter). HTTP is “native” and obvious. It also allows for using things like Gzip encoding to cut down on bandwidth and all other nice things that HTTP allows for. It’s also firewall friendly, something that TCP based solutions often aren’t. XMPP can also be very resource intensive, a supplemental feed really isn’t.
• Would cut down on unnecessary feed polling. As noted by FriendFeed they would still need to poll, but the interval would be significantly higher resulting in less requests.

It’s important to note the difference in computational resources “requests”, “bandwidth”, and “CPU”. It seems a lot of people commenting on the proposal have confused them. This proposal would reduce requests and bandwidth and CPU provided enough consumers supported SUP.

Every time a consumer reads an RSS feed on a dynamic site (assuming no cache exists), the database is being hit to get the latest items. Even if a If-Modified-Since header is sent with the request, the site still needs to check if there’s something newer than that date. For this reason, using If-Modified-Since, while conserving bandwidth doesn’t help much to reduce the number of requests or CPU required. SUP works around this by only hitting one feed, and then only retrieving known updated feeds. Clearly If-Modified-Since and SUP aren’t doing the same thing. They are however somewhat complementary.

To generate this SUP feed, one must essentially do one of two things:

• On a “cron” or some other interval based method query the database for updates and generate a feed. That includes feeds someone may or may not request (think about all those accounts you created on a website and never again visited). Sharded databases may also add more complexity. This can be pretty ugly.
• When a change takes place, for example when a user adds a photo on Flickr, update a table in a database containing the SUP-IDs of changed feeds. Then on interval generate a feed using that one table. Periodically that table needs to be flushed of records no longer needed. Generating the feed off of that table is substantially easier. Of course one can argue if that’s “correct” or not from a data modeling point of view. It’s not really normalized, but than again, how many production databases really are, at least when performance matters?

This isn’t exactly a brand new idea. Six Apart has an Atom based update service, Twitter uses XMPP (though it looks like they may phase that out). LiveJournal at one point had a TCP based system that did essentially the same thing.

Overall SUP isn’t really a bad way of doing things. In a sense, it’s Google Sitemaps for feeds. It solves a problem that today doesn’t have a great solution.

I agree XML is bulky and wasteful for the task. There’s a reason why many web developers prefer JSON rather than actual XML when using xmlHttpRequest: XML parsing can be a real performance killer. JSON in my mind is currently the winner in this department since it’s light weight, simple, and a can be interpreted by pretty much any language on the planet (may need to install a module, gem, extension, or include a class). The downside to JSON is that it doesn’t really allow you to define structure. JSON also is still not binary format, so you have a performance penalty to parse the string. The upside is that JSON is rather easy for humans to read (great for debugging). The NY Times even made a database abstraction layer called DBSlayer that interfaces using JSON.

Serialized PHP has become somewhat popular (Yahoo Developer Network API’s support it), but it’s language specific, though interpreters that can read/write it exist for other languages including Perl, Python and Java. It’s also somewhat complicated for what it provides. At a glance it’s a string of garbage until you break it down.

It looks like Google already has support for Java, Python, and C++. It’s only a matter of time before Perl, PHP, and Ruby get support for Protocol Buffers as well.

I could see Protocol Buffers being pretty useful in combination with Memcached.

It’s great to see Google open sourcing stuff like this.