Netflix is trying to reduce it’s dependency on CDN’s by peering directly with ISP’s and with a new hardware appliance ISP’s can host on their own network to offload traffic. The peering option is pretty strait forward. The appliance however is interesting. Netflix is actually quite transparent about what they are doing, so I thought I’d dig in and take a little look since they are sharing:
Netflix says right up front they were influenced by Backblaze, and their appliance is actually quite similar in many respects. The difference is that Netflix does need a bit more CPU and Network IO and a little less storage. That balance is pretty achievable. The appliance must be a tad on the heavy side as this is a pretty heavily packed server.
Essentially the hardware is a Supermicro mATX board and a bunch of SATA hard drives in a custom 4U enclosure. There are 2 16 port LSI SAS controllers for 32 drives and 4 drives presumably running directly off the motherboard. Hitachi Deskstar or Seagate Barracuda drives. Nothing fancy here. An interesting tidbit is there are 2 x 512 GB “flash storage” (presumably SSD) for logs, OS, popular content. I’d assume those two are running in RAID 0 as one volume. They are managing the spinning disks in software RAID so they can handle failures.
FreeBSD is the OS of choice. Not sure if this software RAID they are doing is something they cooked up or something already out there. Another interesting note is they are using nginx for a web server and are using http for moving content. Huge win for nginx and says a lot for it’s abilities as a web server. It also sounds like Netflix is a customer of NGINX, Inc.
The idea of an appliance on the ISP end isn’t new. CDN’s generally live close, not in the ISP’s network. On the TV side Weather Channel has done this for ages via the little known WeatherSTAR appliances (pic). They sit at the headend and get weather from TWC. They then output local weather reports as a video for the cable provider to insert. The WeatherSTAR appliance like the Netflix appliance is essentially 0 maintenance. It just lives locally and serves it’s master remotely.
It’s nice that they’ve been as open as they have about what they are building. They also have an engineering blog worth keeping an eye on.