Poor Website Email Practices

I got a few emails in the past 24 hours that need to be addressed. I’ve seen both of these issues before, but never has it become so common that I see two almost back to back.

Character Encoding

VideoSurf sent me an invitation to check out their product. Unfortunately I’m a somewhat busy person and just haven’t gotten around to it. They noticed this and sent me a reminder, which I thought was kind of nice. Unfortunately like many companies these days, their mail software doesn’t set a character encoding, meaning their email looks like garbage. If I change the character encoding in my mail client to UTF-8 all looks great. What’s the lesson here?

Content-Type: text/plain; charset=utf-8

That’s all it takes to make sure I see every character in your email. It’s not hard.

Unnecessary Backscatter

Yahoo’s Flickr service sent me an email that my “upload has failed”. I know that’s not true since I don’t use Flickr to host my images. Viewing the email it’s obvious a spammer trying to abuse their service forged the From: header with my email address. This failed for the spammer, and the fail notification went to me. I host SPF records so that recipients mail servers can verify if an email originated from a system that’s authorized to send emails from my domain. Why doesn’t Yahoo check to see if this email they received forged headers? This would obviously be a good way to tell if someone is trying to spam their system, and would stop other innocent victims from getting backscatter.

Google Mail Fail

Found an interesting header when doing some tests with mail filtering:

Received: from qb-out-1314.google.com ([172.21.30.5])
        by mx.google.com with ESMTP id k29si2692710qba.7.2008.09.06.14.48.05;
        Sat, 06 Sep 2008 14:48:06 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning user@example.com does not designate 172.21.30.5 as permitted sender) client-ip=172.21.30.5;
Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning user@example.com does not designate 172.21.30.5 as permitted sender) smtp.mail=user@domain.tld
Received: by qb-out-1314.google.com with SMTP id d5so1543676qbd.6
        for <destination@example.com>; Sat, 06 Sep 2008 14:48:04 -0700 (PDT)

See the problem? Look closely. In particular look at this line:

Received-SPF: softfail (google.com: domain of transitioning user@example.com does not designate 172.21.30.5 as permitted sender) client-ip=172.21.30.5;

Look at that IP. RFC 1918 states the “20-bit block” (172.16/12) is for private internets. Google is softfailing emails because it’s sent through it’s own mail servers. Google’s own SPF record looks like this:

;; QUESTION SECTION:
;_spf.google.com.               IN      TXT

;; ANSWER SECTION:
_spf.google.com.        292     IN      TXT     "v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:209.85.128.0/17 ip4:66.102.0.0/20 ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ?all"

I really don’t understand why Google is doing this. They should have their SPF checker whitelisting mail sent from their own servers. SPF is intended to verify the sender. When sent locally it’s pointless and can only be harmful. They can still do other spam checks.

From what I can tell, this seems to happening about 50% of the time, meaning this is something deployed on some but not all Google clusters.

eBay and banks need to implement SPF and Domain Keys

eBay and banks really need implement SPF, Sender Policy Framework and DomainKeys. There I said it.

I see quite a few Phishing attacks every day. And just about all of them aren’t caught by SpamAssassin. Technically they aren’t spam, so that does make sense. But what bothers me is that this is easy mitigate for many potential victims. If eBay and banks supported SPF and DomainKeys, it would be much easier for a filter to tell if the message is legitimate or not. Check out this sample SpamAssassin header from a eBay phishing email I received:


X-Spam-Level: **
X-Spam-Status: No, score=3.0 required=5.0 tests=BAYES_50,HTML_IMAGE_ONLY_28,
	HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HEADER_CTYPE_ONLY,
	MIME_HTML_ONLY autolearn=no version=3.1.0

That’s really not much in this otherwise pretty bad email. The IP of origin isn’t even in North America (it’s Pacific Rim).

Perhaps it’s time to start a campaign to urge institutions subject to having their name used in these attacks to start using a method like SPF and DomainKeys. A mail provider could then throw out emails that don’t match. Anyone know why they still don’t implement one or both of these methods?

It seems to me they could easily take a giant step to solve the problem. I know Google’s Gmail knows about SPF, and Yahoo knows about DomainKeys. That’s two major email providers right there.

Microsoft pushing Sender ID?

Ok, just when I was starting to think that Microsoft may be changing their ways and trying to act in good faith after them fixing their website the other day. Microsoft starts talking about pushing their sender ID stuff on us. Sender ID is Microsoft’s alternative to the other spam prevention techniques such as Yahoo’s DomainKeys. One problem with Sender ID is the licensing, which has caused organizations like Apache Foundation (who oversee the SpamAssassin project), to nix support for Sender ID. AOL has also also dropped support, and looked towards SPF.

I agree one one of these standards is needed to help prevent spam. Personally I think DomainKeys is the most promising of them all. It’s licensing looks like it will be adequate, and it has a fair amount of backing. Google’s Gmail has apparantly implemented SPF and DomainKeys at this time. I think it’s time for everyone to start looking at following their lead. These two technologies look to be the best. And by implementing them, your mail is more likely to get past spam filters. Microsoft is right, it’s time to start acting. But not with their own proprietary stuff.