traffic

According to the W3C Systeam’s blog, there’s a lot of poorly designed software out there. It’s pretty rare that something has a legitimate need to pull down a DTD in order to work. They should never be requesting it on a very frequent basis. It’s a very cachable asset. The post includes some pretty impressive stats too:

..up to 130 million requests per day, with periods of sustained bandwidth usage of 350Mbps, for resources that haven’t changed in years.

They also make a few requests which really all developers should follow. Here’s my summary:

Cache as much as possible, to minimize your impact on others (not to mention improve your performance).
Respect caching headers
Don’t fetch what you don’t need
Identify yourself. Don’t use a generic UA.
Try not to suck.