Remove Spikes From RRDtool Graphs

I use RRDtool to make graphs on various things I monitor like server stats, network stats and it does a relatively good job. My one (big) complaint is that when you restart you occasionally see these gigantic spikes that completely mess up the data. I’ve even seen spikes larger than what the system can technically handle.

Nobody mentioned there’s a removespikes.pl script (download) that will remove these outliers from your rrds. I put together a quick shell script to make it quick for when I need to run it again:

!/bin/sh
 
for i in /path/to/graphs/rrd/*;
do
        perl removespikes.pl $i;
done;

If you have a ton of graphs a quick shell script to iterate through the directly may be quicker. If you only have a handful like me, no big deal.

Keep the script around for the next time you have spikes to deal with.

802.11n Finalized

802.11n, something I was starting to think would never get beyond draft is now approved. Having suffered through “compliant” 802.11b devices I long ago decided wireless networking is fussy enough to warrant stricter standards. As a result I stuck to Wi-Fi Alliance certified 802.11g devices, and the results have been awesome. I’m still of the opinion that the difference between “compliant” and “certified” is gigantic. Certified 802.11n devices should start to appear in the next few months.

Looks like the goals for any 802.11n upgrade are MIMO (obviously) and preferably dual-band (2.4GHz and 5GHz). I can’t see why I would want to do anything otherwise.

Considering most ISP’s don’t yet provide the downstream or upstream bandwidth necessary to take saturate a good 802.11g network, I’m not sure it’s really necessary to upgrade just yet. Thanks to a solid signal I can sustain up to about 19 Mbps over 802.11g even with WPA2 overhead and slight signal degradation. Under 1ms pings as well. ISP currently offers up to 16 Mbps, 12 Mbps plans for mortals. Rarely is that performance actually seen thanks to “the Internets being a series of tubes”. At least for today upgrading would only improve local network performance, not Internet performance. Most traffic is going outside the network anyway. 802.11n would bring capacity up to 130 Mbps, but since the uplink is still 12 Mbps, that really provides no real performance boost.

For anyone who would argue the faster CPU’s on the newer access points would improve performance, I’ve found that my current AP rarely sees more than a 2% load, with rare spikes up to about 40% capacity.

Of course hardware providers, and retail outlets will continue to tell people that downloading will be 6X faster1, but logic and common sense proves otherwise. It’s the equivalent of a Bugatti Veyron stuck behind a funeral procession.

That of course also assumes all devices are connecting via 802.11n. If you have an 802.11g and 802.11n devices connecting over 2.4 GHz, you’re going to be in mixed mode and slow down while 802.11g devices send/receive anyway. As far as I know there’s no way around that.

Then there’s the issue of all the pre-N adapters sold in laptops over the past few years and their compatibility, which is generally pretty good, but not perfect when mixing vendors.

So despite the marketing getting even stronger, I don’t see how it would be really beneficial to upgrade just yet. The actual performance increase for most activity will be virtually non-existent until ISP’s get faster. I’d rather wait until the hardware matures and prices drop more.

1. up to 6X faster, actual results may vary.

Poor Broadband Performance

For the past several weeks, the cable modem has been getting more and more unstable. Having dealt with this before I knew the signal quality was pretty poor from looking at the stats. By using a different line that goes more direct, it made a real difference as the data below shows (sidenote: I need to start tracking this using RRDtool).

Before

Forward Path:
Signal Acquired at 723.000 MHz
SNR: 27.1 dB
Received Signal Strength: -19.4 dBmV
Bit Error Rate: 0.459 %
Modulation: 256 QAM

Return Path:
Connection: Acquired
Frequency: 31.6 MHz
Power Level: 61.0 dBmV
Channel ID: 4
Modulation: 16 QAM

After

Forward Path:
Signal Acquired at 723.000 MHz 
SNR: 37.1 dB 
Received Signal Strength: -6.9 dBmV 
Bit Error Rate: 0.000 % 
Modulation: 256 QAM 

Return Path:
Connection: Acquired 
Frequency: 31.6 MHz 
Power Level: 49.8 dBmV 
Channel ID: 4 
Modulation: 16 QAM 

The performance before was getting pretty bad (never more than 10Mbps, often below 4Mbps). Just ran another test and got this:

Comcast Speed

You can see the packet loss was at 100% for several hours yesterday, and was even when up the connection was pretty poor. Around 3:00 it was disconnected while they fixed the coax hookup. You can see the clean connection afterward, with only one small hiccup while I made a little adjustment to the networking cabling that resulted in a few minutes down.
Packet Loss

Pings to this server are still a little high after the tornado incident due to some weird routing on Comcast’s part. Not sure when will get resolved.

Nokia N800

Several months ago I mentioned the Nokia N800. Recently a Mozilla based browser was released. This got me extra interested. Apparently flash support is improved, and you could run a better browser. Also can run Skype, and openSSH. What more could you want?

Well there is a killer thing missing: offline support. It’s a WiFi only device (which IMHO is fine), meaning no data connection if your not in range of a hotspot unless you use bluetooth and your cellphone, but that’s expensive. I’d love to see offline support. I’m surprised Google hasn’t stepped up to the plate and released a version of Google Gears the N800. Seems like it would be the perfect application.

Very interesting product.

AirPort Extreme’s Shortcomings

Apple is now shipping the AirPort Extreme. I personally think it’s a pretty nice wireless access point, but it has a few shortcomings which would make me a little hesitant. I’m hoping on the 2nd gen they fix it up a little. To be fair, I haven’t found the perfect Wireless device yet and AirPort Extreme’s shortcomings don’t exactly put it out of the running. For the price I’d expect to see more. Here’s what popped into my mind after reading more about it. Why is this on my mind? Because I just saw some great pictures on unboxing the AirPort Extreme.

  • 10/100 Ethernet – Now 100Mbps is pretty good, but when 802.11n is supposed to reach 540 Mbit/s, I expect Gigabit Ethernet. Especially on a device that expensive.
  • 3 Ports Switch – At first glance you may think the device includes a 4 port hub. The reality is it’s a 3 port hub. The 4th is the uplink (where you plug your modem in). Granted you can get a switch for cheap, it’s not the same. All that money, wireless capacity, and your sharing a wired 100Mbps port? Something is not right.
  • VPN Endpoint – Apple still hasn’t included a VPN Endpoint. Apple includes support for common VPN protocols like L2TP, IPSec, PPTP with Mac OS X for a while, as does Windows. A built in VPN endpoint would be a great addition.
  • Security – Documentation doesn’t mention anything about Stateful Packet Inspection (SPI) or DoS protection. Instead it mentions a vague “NAT firewall”. Not quite sure what that exactly is.
  • Other Features – Also lacking is WMM (Wireless Multi-media) , IGMP snooping, and UpNP (though I don’t care too much about UpNP). If there is support for any of this, it’s not mentioned anywhere I could find. Not even a mention about WDS, which was the most surprising to be missing from the list of acronyms. According to a comment below WDS does exist.
  • It’s not 802.11n certified – Truth is nobody has certification because the standard isn’t official yet. I’d personally like to wait to ensure I get something that is certified.

On a sidenote, did anyone else notice that neither the Airport Extreme website, nor the Apple 802.11 page give any numbers in regards to 802.11n performance? It won’t say more than “Up to five times the performance and up to twice the range compared to the earlier 802.11g standard.” I found it very strange to see no numbers “up to XMbps”.

Nokia N800 Looks Interesting But Large

The Nokia N800 looks really cool, but I wonder if it’s size is a little to large to make it practical to carry around. The lack of good Flash support, and weak MPEG4 (my guess would be lack of an onboard decoder and using software to make up for it). The fact that it’s Linux powered means you can run a fair amount of desktop software on it, with a little patience.

The size is really what hurts. If it were a little smaller, it would be a really cool PDA-like device.

I’m still curious what people will figure out can be done with the hardware. It’s still too new to tell. From what I’ve read it’s somewhat powerful, so the potential is there.

Norton “Internet Worm Protection”

Norton AntiVirus has this strange omission I just can’t figure out. For some reason “Internet Worm Protection” won’t allow for creating a connection to a PPTP VPN. Not very helpful if you have to connect to one of the many VPN’s out there that use this protocol.

First a little primer on making a PPTP connection . You essentially need two ports open, 1723/TCP, and IP Protocol 47 (GRE). Ok, this is pretty basic stuff. We can do that ;). Well in the little wizard Norton provides, to create a rule you have the following choices for protocol: TCP, UDP, TCP/UDP, ICMP, ICMPv6, All (pointless). No way to select GRE.

So the only way I’ve found to connect to a PPTP VPN thus far is simply to disable either just Internet Worm Protection, or disable Norton AV.

It’s rather odd that something like this is not supported. A search on Google didn’t turn up an answer. Symantec’s tech support database didn’t turn up anything helpful either.

I would have expected something like this to function without a hitch. I’m very surprised to see this requires any intervention, and even more surprised to see that even with intervention there’s still no way to get it working.

NSA… Can you hear me now?

Of course when the Federal Government uses the legal equivalent to the Atomic Bomb, you know (despite their insistence) that all of the alleged activity is true… otherwise they would defend it. They even went as far as stating:

The fact that the United States will assert the state secrets privilege should not be construed as a confirmation or denial of any of plaintiffs’ allegations, either about AT&T or the alleged surveillance activities.

Yea sure. You don’t invoke something like this when you have nothing to hide. It’s like how all those companies “settle” but don’t admit guilt or wrongdoing. You don’t pay for something you didn’t do.

By the way, if you traceroute to this website and see “att.com” anywhere in there, you can rest assured they know your reading this ;-).

Save the Internet

Save The Internet

I’ve spent quite a bit of time encouraging browser choice and platform choice by strongly advocating and encouraging compatibility.

It’s sad but it’s now time to do advocate the right to choose what websites and/or services you want to use. If you enjoy using:

Search engine of your choice
VoIP provider of your choice
Music/Video provider of your choice
News provider of your choice

Boy that’s a lot of links, yet just a handful of the many available… you get the idea. You care.

I don’t even want to think of the impact on open source and innovation if every website has to pay for decent performance (imagine degraded downloads of linux iso’s simply because the distro isn’t paying your specific ISP).

I’m pretty sure I’m not alone in enjoying the freedom of choice the Internet provides.