DBSlayer + Node.js

Lately it seems the rage among developers is to take node.js and combine it with something else unusual. So here’s my contribution.

DBSlayer is a project by NYTimes a few years ago that seems to be somewhat forgotten but is pretty cool. It’s another MySQL proxy, but with a slight twist. Rather than use a binary protocol, or XML, they went with JSON. It supports things like connection pooling, round-robin distribution to slaves, automatic fallover, and it’s mutithreaded. It’s pretty fast and easy to work with. It’s almost like turning a MySQL database into a REST API. You pass a SQL query as a query argument and it gives you a JSON response.

Once you start it you can do something query using a request like:

http://localhost:9090/db?%7B%22SQL%22%3A%22SELECT%20*%20FROM%20facts%20WHERE%20id%3D1%3B%22%7D%20

That will give you a JSON object containing the result of your query.

So doing that in node.js is roughly:

var sql = ‘{"SQL":"SELECT * FROM facts WHERE id=7;"}’;
 
var http = require(‘http’);
var client = http.createClient(9090, ‘localhost’);
 
var request = client.request(‘GET’, ‘/db?’ + escape(sql), {});
request.end();
request.on(‘response’, function (response) {
  console.log(‘STATUS: ‘ + response.statusCode);
  console.log(‘HEADERS: ‘ + JSON.stringify(response.headers));
  response.setEncoding(‘utf8’);
  response.on(‘data’, function (chunk) {
    console.log(‘BODY: ‘ + chunk);
  });
});

Running that looks like this:

$ node test.js
STATUS: 200
HEADERS: {"date":"Fri, 27 May 2011 02:02:27 GMT","server":"dbslayer/beta-12","content-type":"text/plain; charset=utf-8","content-length":"290","connection":"close"}
BODY: {"RESULT" : {"HEADER" : ["id" , "fact" , "author" , "ip" , "timestamp"] , "ROWS" : [[7 , "1+1=2" , "raccettura" , "127.0.0.1" , 123456]] , "TYPES" : ["MYSQL_TYPE_LONG" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_VAR_STRING" , "MYSQL_TYPE_LONGLONG"]} , "SERVER" : "db"}

You could obviously clean that up and create a little library to hide the HTTP parts.

It’s an interesting JS centric way to abstract your database while maintaining SQL level control. JSON is becoming the new XML.

On Apple’s Location Tracking

The controversy over Apple’s “Location Tracking” is quite interesting. It’s worth making clear that the nodes stored in the database are approximations of cell phone towers and WiFi hotspots you’re likely to encounter rather than your location(s) at any given point in time. It’s a way to “prime the well” when doing a GPS lookup to improve performance.

Apple notably failed in a few key ways which should serve as a lesson to others:

  1. Always disclose what you’re doing. – Never just assume what you’re doing with someone’s information is cool. Apple could have mitigated a lot of this had they disclosed what the phone was actually doing from day 1. Never transmit anonymous or personal information without letting the user know first.
  2. Never store more than you need – I can’t believe how many companies mess this up. Storing user information is a liability. A good business limits it’s liabilities to only what’s necessary to conduct business. Storing so much data, and not expunging was a very bad move and amplified the situation. On top of not letting users know what was going on, there was no way to purge information. This just made things much worse. Apple went as far as backing up what should be an expendable cache.
  3. Always be paranoid with information – Apple states “The local cache is protected with iOS security features, but it is not encrypted. Beginning with the next major release of iOS, the operating system will encrypt any local cache of the hotspot and cell tower location information.” in the response to Edward J. Markey. This should have been encrypted since day 1. Various tools existed for a few years that could read this data in the surveillance community. Apple undoubtedly knew people were using this data sometimes for illicit purposes. No company has gotten in trouble for being to secure with customer information with anyone other than the NSA or FBI.

It’s worth noting that their software update in response to this controversy is actually pretty good and pretty thorough. I’m surprised they couldn’t quickly shim some encryption around it. The iOS is loaded with enough DRM and crypto.

On another note, I fully expect some court cases to be reopened now that “cell phone records” are not quite as accurate as they were falsely billed to be. Also companies who marketed software are capable of showing a users location history may be liable as this wasn’t accurately vetted. If they did good testing they would have seen the extent of it’s “tracking”. It seems inevitable.

Lastly, I wonder how much battery life, and how much bandwidth this was utilizing. Some customers are on metered WiFi (especially some hotspots). To geo-tag one must turn on GPS, meaning battery life was being drained behind the scenes.

Apple’s full response can be found on Congressman Ed Markey’s website (copied here for perpetuity).