Year: 2010

Categories
Networking

DNS And CDN Performance Implications

I’ve seen various people complain about performance problems when using services like Google’s DNS or OpenDNS. The reason why people generally see these problems is because many large websites live behind Content Distribution Networks (known as a CDN) to serve at least part of their content, or even their entire site. You’re getting a sub-optimal response and your connection is slower than needed.

I’ve worked on large websites and setup some websites from DNS to HTML. As a result I’ve got some experience in this realm.

How DNS Works

To understand why this is, you first need to know how DNS works. When you connect to any site, your computer first makes a DNS query to get an IP address for the server(s) that will give the content you requested. For example, to connect to this blog, you’re computer asks your ISP’s DNS servers for robert.accettura.com and it gets an IP back. Your ISP’s DNS either has this information cached from a previous request, or it asks the websites DNS what IP to use, then relays the information back to you.

This looks something like this schematically:

[You] --DNS query--> [ISP DNS] --DNS query--> [Website DNS] --response--> [ISP DNS] --response--> [You]

Next your computer contacts that IP and requests the web page you wanted. The server then gives your computer the requested content. That looks something like this:

[You] --http request--> [Web Server] --response--> [You]

That’s how DNS works, and how you get a basic web page.

How a CDN Works

Now when you’re website gets large enough, you may have servers in multiple data centers around the world, or contract with a service provider who has these servers for you (most contract). This is called a content distribution network (CDN). Parts of, or your entire website may be hosted with a CDN. The idea is that if you put servers close to your users, they will get content faster.

Say the user is in New York, and the server is in Los Angeles. You’re connection may look something like this:

New York : 12.565 ms  10.199 ms
San Jose: 98.288 ms  96.759 ms  90.799 ms
Los Angeles: 88.498 ms  92.070 ms  90.940 ms

Now if the user is in New York and the server is in New York:

New York: 21.094 ms  20.573 ms  19.779 ms
New York: 19.294 ms  16.810 ms  24.608 ms

In both cases I’m paraphrasing a real traceroute for simplicity. As you can see, keeping the traffic in New York vs going across the country is faster since it reduces latency. That’s just in the US. Imagine someone in Europe or Asia. The difference can be large.

The way this happens is a company using a CDN generally sets up a CNAME entry in their DNS records to point to their CDN. Think of a CNAME as an alias that points to another DNS record. For example Facebook hosts their images and other static content on static.ak.facebook.com. static.ak.facebook.com is a CNAME to static.ak.facebook.com.edgesuite.net. (the period at the end is normal). We’ll use this as an example from here on out…

This makes your computer do an extra DNS query, which ironically slows things down! However in theory we make up the time and then some as illustrated earlier by using a closer server. When your computer sees the record is a CNAME it does another query to get an IP for the CNAME’s value. The end result is something like this:

$ host static.ak.facebook.com
static.ak.facebook.com is an alias for static.ak.facebook.com.edgesuite.net.
static.ak.facebook.com.edgesuite.net is an alias for a749.g.akamai.net.
a749.g.akamai.net has address 64.208.248.243
a749.g.akamai.net has address 64.208.248.208

That last query is going to the CDN’s DNS instead of the website. The CDN gives an IP (sometimes multiple) that it feels is closest to whomever is requesting it (the DNS server). That’s the important takeaway from this crash course in DNS. The CDN only sees the DNS server of the requester, not the requester itself. It therefore gives an IP that it thinks is closest based on the DNS server making the query.

The use of a CNAME is why many large websites will 301 you to from foo.com to www.foo.com. foo.com must be an A record. To keep you behind the CDN they 301.

Now lets see it in action!

Here’s what a request from NJ for an IP for static.ak.facebook.com looks like:

$ host static.ak.facebook.com
static.ak.facebook.com is an alias for static.ak.facebook.com.edgesuite.net.
static.ak.facebook.com.edgesuite.net is an alias for a749.g.akamai.net.
a749.g.akamai.net has address 64.208.248.243
a749.g.akamai.net has address 64.208.248.208

Now lets trace the connection to one of these responses:

$ traceroute static.ak.facebook.com
traceroute: Warning: static.ak.facebook.com has multiple addresses; using 64.208.248.243
traceroute to a749.g.akamai.net (64.208.248.243), 64 hops max, 52 byte packets
 1  192.168.x.x (192.168.x.x)  1.339 ms  1.103 ms  0.975 ms
 2  c-xxx-xxx-xxx-xxx.hsd1.nj.comcast.net (xxx.xxx.xxx.xxx)  25.431 ms  19.178 ms  22.067 ms
 3  xe-2-1-0-0-sur01.ebrunswick.nj.panjde.comcast.net (68.87.214.185)  9.962 ms  8.674 ms  10.060 ms
 4  xe-3-1-2-0-ar03.plainfield.nj.panjde.comcast.net (68.85.62.49)  10.208 ms  8.809 ms  10.566 ms
 5  68.86.95.177 (68.86.95.177)  13.796 ms
    68.86.95.173 (68.86.95.173)  12.361 ms  10.774 ms
 6  tengigabitethernet1-4.ar5.nyc1.gblx.net (64.208.222.57)  18.711 ms  18.620 ms  17.337 ms
 7  64.208.248.243 (64.208.248.243)  55.652 ms  24.835 ms  17.277 ms

That’s only about 50 miles away and as low as 17ms latency. Not bad!

Now here’s the same query done from Texas:

$ host static.ak.facebook.com
static.ak.facebook.com is an alias for static.ak.facebook.com.edgesuite.net.
static.ak.facebook.com.edgesuite.net is an alias for a749.g.akamai.net.
a749.g.akamai.net has address 72.247.246.16
a749.g.akamai.net has address 72.247.246.19

Now lets trace the connection to one of these responses:

$ traceroute static.ak.facebook.com
traceroute to static.ak.facebook.com (63.97.123.59), 30 hops max, 40 byte packets
 1  xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx)  2.737 ms  2.944 ms  3.188 ms
 2  98.129.84.172 (98.129.84.172)  0.423 ms  0.446 ms  0.489 ms
 3  98.129.84.177 (98.129.84.177)  0.429 ms  0.453 ms  0.461 ms
 4  dal-edge-16.inet.qwest.net (205.171.62.41)  1.350 ms  1.346 ms  1.378 ms
 5  * * *
 6  63.146.27.126 (63.146.27.126)  47.582 ms  47.557 ms  47.504 ms
 7  0.ae1.XL4.DFW7.ALTER.NET (152.63.96.86)  1.640 ms  1.730 ms  1.725 ms
 8  TenGigE0-5-0-0.GW4.DFW13.ALTER.NET (152.63.97.197)  2.129 ms  1.976 ms TenGigE0-5-1-0.GW4.DFW13.ALTER.NET (152.63.101.62)  1.783 ms
 9   (63.97.123.59)  1.450 ms  1.414 ms  1.615 ms

The response this time is from the same city and a mere 1.6 ms away!

For comparison www.facebook.com does not appear to be on a CDN, Facebook serves this content directly off of their servers (which are in a few data centers). From NJ the ping time averages 101.576 ms, and from Texas 47.884 ms. That’s a huge difference.

Since www.facebook.com hosts pages specifically outputted for the user, putting them through a CDN would be pointless since the CDN would have to go to Facebooks servers for every request. For things like images and stylesheets a CDN can cache them at each node.

Wrapping It Up

Now the reason why using a DNS service like Google’s DNS or OpenDNS will slow you down is that while a DNS query may be quick, you may no longer be using the closest servers a CDN can give you. You generally only make a few DNS queries per pageview, but may make a dozen or so requests for different assets that compose a page. In cases where a website is behind a CDN, I’m not sure that using even a faster DNS service will ever payoff. For smaller sites, it obviously would since this variable is removed from the equation.

There are a few proposals floating around out there to resolve this limitation in DNS, but at this point there’s nothing in place.

Categories
Photos

A Festivus Pilgrimage

H&H Bagels Exterior

For those who don’t know, the Festivus holiday is from an episode of Seinfeld titled “The Strike”. The other plot of the episode is Kramer ending a 12 year strike at H&H Bagels where he used to work. H&H Bagels is a NYC institution that actually exists on the upper west side. What better way to celebrate Festivus than togo to H&H Bagels? I couldn’t think of one.

The show:

H&H Bagels - Seinfeld

The real place:

H&H Bagels - Seinfeld
H&H Bagels Zagat Rating
H&H Bagels The Sandwich Chronicicle with Jerry Seinfeld
H&H Bagels Exterior

I haven’t been to many of the popular bagel establishments in NYC to compare, so I can’t say for certain if they are the best in the city or not. I will say it was very good and served very fresh. It’s also very likely the first Bagel place I’ve been to with a Zagat plaque on the wall.

This may be a new holiday tradition.

Categories
In The News

Common Application Bugs

There’s a curious article in the NY Times about the Common Application‘s technical glitches. The Common Application is a uniform way of filling out one application to apply to many colleges, as opposed to filling out an application for each individual college.

As a web developer, this struck me as particularly odd:

As it turns out, applicants do not have, say, 150 words to discuss their most meaningful extracurricular activities; they have something closer to 1,000 characters (Max said he eventually figured this out). And because some letters may take up more space than others, one applicant’s 145-word essay may be too long, while another’s 157-word response may come up short, Mr. Killion said.

“A capital W takes up 10 times the space of a period,” he said. “If a student writes 163 characters that include lots of Ws and m’s and g’s and capital letters, their 163 characters are going to take many more inches of space than someone who uses lots of I’s and commas and periods and spaces.”

Asked why the problem had not been fixed, Mr. Killion said, “Believe me, if there’s a way to do it, we’d do it. Maybe there’s a way out there we don’t know about.”

Sounds like the folks behind the common application need to go back to middle school and learn about variable width and fixed width fonts. If they had switched to fixed width fonts in the <textarea/> and used the same number of cols and font size it should be pretty accurate. I’m guessing some designer insisted on Helvetica or whatever they are using.

That said, are they actually printing these things out? Is there no way to do this electronically in 2010?

Back in my day (2002), I was advised to go the paper route since many still felt/feared that electronic applications weren’t being fairly considered (and in some cases not processed correctly). That was also the first year public colleges could join the Common Application as I recall. I suspect I was the last class where the majority did it on paper. I guess it’s still an improvement for a technophobic educational system.

Categories
Around The Web

Remember: Winning Isn’t Normal

Winning Isn’t Normal is a great blog post by Jason Shen about just how much goes into being good at something. Particularly great is this quote:

Remember one thing… winning isn’t normal. That doesn’t mean there is anything wrong with winning. It just isn’t the norm. It is highly unusual … So it requires unusual action. In order to win, you must do extraordinary things.

It seems too many people forget that, and I’m certain I’m no exception to that. Putting intelligence and innate gifts aside (they only get you so far), being good at something to the point of earning any respect takes an incredible amount of effort, sacrifice and time. The old standard for being viewed as an expert was 10,000 hours of experience or 10 years. I suspect in many modern-day professions that’s way below standards.

It’s way too easy to look at the end results and overlook what goes into something. From that 2 minute Olympic performance to the 5 minute medical diagnosis etc. Even from a purely academic standpoint we rarely realize what goes into anything other than a law or medical degree, and even there most people don’t get the late nights and dragging yourself through hell it takes. Most just think about sitting through the Bar Exam and MCAT’s like another SAT and not the prep that goes into it.

Categories
Around The Web Audio/Video Funny

Waka Flocka Flame No Hands – Acoustic Cover

DannyVola - No Hands - Waka Flocka Flame Parody

DannyVola - No Hands - Waka Flocka Flame Parody

I’m going to start by saying NSFW. You will want headphones if listening to this video at work or in the presence of easily offended. It’s extremely well done. It’s based on this song, which is obviously also NSFW.

If it’s not already obvious, I have a love for musical parodies and oddly creative covers. I’ve got a few more in queue for this blog.

Categories
Blog General

My Lack Of New Blog Posts

This blog has been quiet for a longer period than normal while I’m working on some interesting projects of various scopes. More to be revealed eventually. I don’t anticipate this silence lasting too much longer.

Categories
Comic

Comic: Men’s Room Zones

Men's Room unfasten pants zone

Men's Room unfasten pants zone

Categories
Around The Web

Intelligence And Staying Up Late

London School of Economics and Political Science research as summarized by the Winnipeg Free Press:

People with higher IQs are more apt to be nocturnal night-owls. Those with lower IQs tend to restrict their activities primarily to daytime.

Of course it’s important to note correlation does not imply causation. That said, lots of genius work does happen between 10 PM and 4 AM. There are countless stories of overnight success and innovation. I can’t recall many stories of 9 AM eureka moments unless they were the conclusion of overnight work.

Categories
Around The Web

Changing Education Paradigms

Educational Paradigms

Educational Paradigms

Another great RSA talk by Sir Ken Robinson animated, it’s well worth the 11 minutes to watch. It’s hardly a secret how poor the education system really is and how many it fails.

I should note the intro video on his website has perhaps one of the greatest quotes I’ve heard in a while: “If you’re not prepared to be wrong, you’ll never come up with anything original”.

Categories
Around The Web Audio/Video

Microsoft’s “Really” Ad

Microsoft "Really" Ad - Urinal

Microsoft "Really" Ad - Urinal

I really like this ad. Over the past 3 years or so this is what America has become.

Truthfully, I try not to use my phone when walking, talking, using the bathroom, at the table or other situations where it just seems grossly inappropriate. I think I’m one of the last holdouts. Given my inclination to prefer gadgets to social norms, this shows how far it’s gone.

Given this behavior skews even younger than me, I fear the worst is ahead of us.