Inside Google’s Data Centers

Google Data Center Storm Trooper

Google’s opening up about their data centers in a pretty big way. From being secret to even the locations a few years ago they’ve now posted a street view tour, as well as some pretty great video. Facebook has also become a bit more open in terms of their data center operations.

Part of this openness is to make the “internet” seem more trustworthy and less intimidating. The other part is to show off the energy-saving improvements they are making in the wake of controversy data centers have faced over their power usage.

I think someone at Google or Facebook needs to get me a tour of their facility 😉 .

Google’s Data Center Secrets

Big news today is that Google “unveiled” (more like confirmed) some data center secrets:

Custom Servers

It has been known for years that Google has been building it’s own servers rather than buy from a vendor. They have defended this as their servers are more efficient and customized for their needs than they could ever buy. They cut out things like a video card which do nothing but add a point of failure and waste power. They put a battery on the server itself rather than have a UPS for the rack they found it to be more cheaper and more efficient. They also hang the power supply away from the rest of the system itself, presumably for cooling. This actually isn’t shocking since it’s been leaked several times before, though this is the first time that I’m aware of Google speaking publicly about their design in this much detail.

Container Data Centers

Apparently since 2005 Google has been using shipping containers as data centers. It’s been known for a long time Google was interested in the idea (as were other companies) but a first that they have actually been using them for a while. 1,160 servers per container utilizing 250 kilowatts of power = 780 watts per square foot. Very impressive.

I guess it’s only a matter of time before we see commercial servers, and perhaps even some desktops with power supplies that have their own batteries.

Update [4/11/2009 @ 5:00 PM EST]: Google has a blog post up including video of the summit.

Slow Site

Last Friday (May 2), the data center where this site lives suffered a power fluctuation due to some tornado activity in the area. The actual outage (if there was even one) seemed to have been in the 5 minute ballpark based on various monitors. Apparently this somehow resulted in a routing problem resulting in some lag and packet loss for some (including myself). Possibly a router that didn’t persist as well as one would hope. This is being investigated.

As a result, if this site (and it’s feed) seems slower than normal, that’s the reason.

EnergyGuide For Computers And Servers

Energy Star Energy GuideMost appliances sold in the US are now required to ship with the infamous EnergyGuide Label. That yellow label is most associated with major appliances, but it could be utilized beyond that. It’s a pretty simple idea in principle. The label makes it very clear how much power the device consumes, and based on average usage how much it will cost the consumer to operate on a yearly basis.

Perhaps it’s about time to adopt a similar convention for computers, servers, displays, printers, and networking equipment. Doing so would have several benefits:

  • Make consumers aware of the variety in efficiency – Different products consume different amounts of power. Being able to easily compare isn’t a bad thing.
  • Encourage manufacturers to be more competitive – While things such as processors have become more efficient (Pentium D consumption:performance ratio was worse than say the Core 2 Duo), computer manufacturers aren’t really pressed to adopt the most power efficient technologies since they may increase component costs.
  • Weed out poor power supplies and power bricks – A poorly kept secret is that power supplies and those power bricks we all have under our desk aren’t the most efficient devices out there. There are however some newer ones that have improved greatly like 80 PLUS certified power supplies. By making power consumption more visible, the really cheap power supplies become a negative in computing rather than a silent component.
  • Lower costs for businesses – It’s no secret businesses (like google) are trying to lower costs in their data centers. By making power consumption more obvious, it would be easier for them to make better choices to reduce their power costs and get the most out of their budget.

The real problem is that it’s tough to go to a manufacturer’s website or a store and tell what this device will cost you to operate, and compare to another device. That doesn’t really seem necessary. It should be easy to tell.

I think such a label should include the following:

  • Estimated Yearly Electric Use – in kWh for 24×7 power consumption pattern, additionally 8×5 for desktop computers, non-networked printers, displays. For 8×5 should also show the estimated yearly electric use in standby for the remainder of the time. That’s 1 number for servers, network equipment (which operate 24×7), maximum of 3 for everything else.
  • Estimated Yearly Operating Cost – in $ based on the above kWh data.
  • Standby Power Consumption Use – How many watts the device uses when in standby. This is really to see how close it comes to meeting the one watt initiative.
  • Standby Activation Modes – Several badges indicating how standby is initiated. Including: idle timeout, user initiated, timer, auto-wake (comes out of standby when device is needed). These let the consumer know how the device can minimize power consumption.
  • Upgradable Power Supply – Is the power supply connected to the device using an industry standard so that it could be replaced/upgraded during the device’s lifetime?

The data should be made available by easily removable signage attached to the device on shipping, made available on request in stores, and available on the manufacturers website linked from the product page.

By making power consumption more obvious, it would be easier to make a decision based on TCO (total cost of ownership) that accounted for power consumption. The bigger advantage is that it would encourage manufacturers to consider power consumption as a potential way to sell a product, and that means improving power supplies and standby consumption.

It seems like an easy way to spur some innovation and allow for better decisions by consumers. Considering the cost of power is expected to rise, and the need for IT is rising, it seems like a decent proposal. Making the most of the watts available requires knowing consumption. That way keeping servers and desktops powered on is as efficient as possible.

Image from FTC