Tag Archives: server

New Home Server

Posted on by
Reply

Over the past few weeks, I’ve been in the process of setting up a new home server. The previous one was an old Beige G3 (266MHz) running Mac OS X 10.2 that was starting to show it’s age. The new system is a much more capable B&W G3 (400MHz) running Mac OS X 10.4. Despite only a slight increase in clock speed, the B&W G3 has much more modern hardware (USB, Firewire) not to mention more room for more storage. The opportunities are endless.

Decided to go with a multi-drive setup considering the extra bays. The system had a still usable 40GB Seagate Barracuda IV drive which would make a perfect system disk for OS/Software. Installed via a ACard ATA/66 controller it’s no speed daemon, but for the purpose it’s fine. For the data drive I decided to get a SIIG SATA card and a pair of Seagate SATA drives I found a good deal on at BestBuy. The drives were Seagate ST303204N1A1AS, which corresponds to 320GB. Inside the boxes as expected were (the newer and better) ST3320620AS, which is a Seagate Barracuda 7200.10 with firmware 3.AAE (not the AAK people have had in the past). Perfect.

Next I wanted to replicate data across the drives on a cron. Initially I was thinking rsync, since as of 10.4, it’s resource-fork aware. It turns out that’s not really true. I ended up going back to SuperDuper to copy between the drives. It only copies changed files, and once a week will delete removed files (so if you accidentally delete something, there’s still a chance to recover, unless you do it at the wrong time). Not a bad solution IMHO. Still would prefer rsync more. Initial backup took less than 1/2 hour. Just a few minutes should be enough to keep the disks in sync. I briefly considered setting up RAID, but decided against it since RAID is not backup. It doesn’t protect against things like corruption.

Apple needs to kill off resource forks ASAP. They should have done so when moving to Mac OS X several years ago.

Next up, I tried putting a copy of TechTools Pro I no longer use on my Mac Mini since upgrading to Leopard on the system, but that resulted in some drive problems that I couldn’t resolve without uninstalling. They seem to know about the problem, but haven’t fixed it. You see the following error repeatedly in the system.log file until you reboot:

kernel[0]: IOATAController device blocking bus.

Drag.

Also updated mrtg, and this time compiled GD, libpng, libjpeg, etc. all by hand, rather than use fink. Last time I went with fink, which saved me a few keystrokes, but when fink no longer updated packages for 10.2, left me high and dry. This time I think I’ll avoid it when possible. I need to try getting RRDtool setup at some point, since it’s so much better.

I use a few php scripts for easy admin of the box, and decided PHP 4 wasn’t adequate since it’s pretty much discontinued. So I upgraded to php 5.2, and all seems good so far. I think Apache 1.3.33 will serve me just fine for the moment, so not upgrading that.

I might give setting up BIND a try, since local DNS would be pretty handy for easily accessing the server without modifying the host file on computers.

I also disabled things like spotlight, which have absolutely no purpose on this box.

On another note, glib for some reason won’t compile for me. No clue what’s going on. Overall it’s looking pretty good. Should be about ready for real use. Just want to make sure the backups work as expected.

Virtualization For Mac OS X?

Posted on by
6

Virtualization is a great way to improve reliability, take advantage of hardware and scale. For example Mozilla’s build team uses it to manage all the build instances that used to be on individual machines. These servers essentially compile code all day long. One problem with virtualization and cross platform building is that Mac OS X doesn’t run in any virtualization environment (because of Apple’s interest in selling hardware). This means while you can run Windows and Linux on the same boxes, you still need to have and maintain separate Xserve’s for the purpose of compiling for Mac OS X. Looks like Mac OS X Server 10.5 (and only server edition) now has a license that permits running virtual. While great, this makes it pretty expensive to do things like a build farm. You can’t just buy a Mac OS X client, even though that’s all you really need. You need to buy server.

Currently, there’s nothing other than PearPC that can run it (and PearPC is worthlessly slow). Hopefully VMWare will update at some point to support it. At that point, things can get interesting.

FoxTorrent

Posted on by
8

I said back in 2004 that Firefox needs built in support for BitTorrent. My idea was it would be integrated into the download manager so that it was “just another protocol” and would be transparent to a typical user. I still stand by that.

Fast forward to 2007: FoxTorrent is by RedSwoosh (now owned by Akamai).

I’d personally love to see something like this ship built in. It’s a great feature. BitTorrent is a great protocol for distributing large downloads without having to buy expensive infrastructure. Akamai’s interest is proof of that.

FoxTorrent has a blog if you want to keep an eye on it. FoxTorrent is MIT licensed as well. It seems like a very interesting product. I’ll have to dig into this and look at it a bit closer.

[Hat tip: TechCrunch]

Yahoo Goes Green

Posted on by
Reply

Yahoo is going carbon neutral. I’m curious how much is offset, and how much is reduction. Yahoo has a fairly large infrastructure. I wonder if they are using alternative power sources, or if they are going to plant a million trees. They do mention:

These projects could include a wind farm in India or a small-scale run of the river hydroelectric project in Brazil. We’re also looking to invest in emerging clean technologies.

Interesting. I wonder if we will see things like carbon neutral VoIP, carbon neutral bandwidth, carbon neutral data centers / colocation / hosting?

Site Backups And Bandwidth Fun

Posted on by
Reply

I keep regular backups of everything on this server just in case something happens. Recently I switched to a more automated and secure (PGP encrypted) solution for this blog due to it’s fast-paced nature. Just the critical stuff (database, media, templates). I choose PGP (implemented using GPG) since it’s easy, and I only have to store the public key on the server, making it safer than most alternatives.

I’m strongly considering moving it all eventually over to Amazon’s S3 storage. At $0.15 per GB-Month of storage used and $0.20 per GB of data transferred it would be very affordable to keep backups in an even more secure fashion. I’d still use my own encryption on top of theirs for extra security. For things like media, I could even see myself hosting it solely at Amazon. It just seems like that may be a more practical and scalable approach.

Unfortunately until either FTTH or DOCSIS 3.0 comes to town, it doesn’t look like Amazon’s S3 will be practical for home backup purposes. This server has a beefy connection to a few large pipes to the internet (Level3, Global Crossing, and Cogent last I checked). They provides high speed connectivity so a backup would take only a few seconds. At home with a cable modem on a DOCSIS 1.1 network (such as Comcast) the bandwidth is just to slim to allow enough upload capacity. Comcast still only allows 384kbps up. Even the top plans in select areas don’t top 1Mbps. Of course these are Comcast’s numbers (the actual performance is often less). In areas that they currently serve, Verizon FiOS (FTTH) is available at 15 Mbps/2 Mbps. Much better suited for such purposes (though more would be welcome). As strange as it may seem pricing is quite competitive, giving cable a run for it’s money. Perhaps one day DOCSIS 3.0 will appear, though that seems to be a while away. Perhaps one day all homes will have 100Mbps full duplex connections with low latency.

The only real way to get around this limitation is to perhaps use rsync to perform backups. Initial backups would still suck, but after that it wouldn’t be too bad. Though that wouldn’t work with services such as Amazon’s S3, which are token based. There is an rsync-like clone, but it’s still not the real thing. Perhaps Google’s upcoming GDrive will be cool enough to allow the use of rsync over SSH (I could dream) in addition to WebDAV (which is what I expect to see). Last I checked rsync doesn’t support WebDAV because WebDAV is done over HTTP. If I understand it right, RFC 3229 would add Delta encoding support to HTTP, making something like rsync over WebDAV possible since it uses delta encoding.

2 failures in a week

Posted on by
Reply

Last week my private server had some corruption on the system partition. Seems to be related to the system log file from what I can figure out. I’ve got it back up and running, mostly. Though not 100%. If it was running something newer than 10.2 Jaguar, I’d likely be in better shape. Not much work, but because it’s 266MHz, it takes a year to do something as simple as upgrade perl. Despite that, the tough little box is still chugging away. Thanks to partitioning, and putting the drive in an external enclosure and hooking it up to my Mac Mini with Mac OS X 10.4, all user data is intact. That’s really what’s important.

Now today a Windows XP system decides to corrupt itself. Not quite done with evaluating the damage and repairs. I got it to boot, not sure what else is hiding under the murky waters of Windows XP. That’s the goal for tonight.

What did I learn? I really need to get some better backup systems working on these two computers. I’m sick of doing this.

More on the Mini

Posted on by
Reply

I mentioned it the other day. Now it looks like someone made a business plan out of it. Pretty cool idea. You could fit a few clients into 2U space. And cost effective.

Perhaps VPS (Virtual Private Servers) are a waste of time. Maybe this approach is better. This way, less clients are effected per hardware outage. This minimizes liability.

I’m 6/10 on getting a Free Mac Mini. I need 4 other people to make my goal of a free Mac Mini come true. So if you want to help your faithful blogger out, signup and complete an offer. most are painless. Read through them and find one that works for you.

No, I don’t currently plan to host on mini’s. But I do hope to have a mini to play with. My Macs are aging, so having a more powerful one for things like building Firefox, Thunderbird. And general development purposes would be really sweet.

So if you can, help Robert out! 4 More!

Mac Mini – Make Your Own Cluster?

Posted on by
2

Ok, I admitted to being a geek a long time ago, but here’s what I’m thinking:

Take Apple’s new Mac Mini, which is 2″ in height (slightly over 1U). Put several of them on a metal tray. Why? Because it can become a makeshift blade server. Granted your limited to Firewire, USB, and 10/100 Ethernet. Regardless, for rather low cost, you can get some decent power. All with 2U of rack space. IMHO that’s pretty cool. Looks like it’s a pretty simple way to get a cluster. You could even use it for some decent web hosting. 1 as a DB server, 1 or 2 as an Apache server, and perhaps a squid proxy. Mail, DNS. Each could have their own redundancy. All in 2U of rack space.

That’s amazingly low cost for having so many CPU’s, and being able to spread out your load on several physical systems.

If I had the cash, I’d give it a go. :-D

Edit: You can help me get one by signing up and completing 1 offer. They are legitimate, I got a free ipod from them not to long ago.