AVG really needs to fix their “LinkScanner” product. It essentially scans pages for links and pre-downloads them to check for malware. If that doesn’t sound so bad, then your obviously not paying for bandwidth or trying to keep your server load manageable. Essentially it means more traffic pegging servers and downloading pages, but most of it being a total waste.
This isn’t just bad for webmasters. This excess traffic hogs ISP’s (who now plan to charge by-the-byte) and WiFi. In a country where we are tight on bandwidth, this is really a pretty lousy implementation.
AVG even went so far as to use multiple user agents, all of which seem to spoof IE, making it more difficult to block.
The best way to block the bogus AVG traffic seem to be by looking for the
Accept-Encoding HTTP header, which could be done using an Apache rewrite rule if you can’t do so on the firewall or load balancer level.
AVG really needs to reaccess this poorly designed product. It’s unnecessarily taxing the web.