It’s pretty well known at this time that a rather large sum of Spam comes through regular ISP’s. There is a rather large debate on how to get rid of them. Some ISP’s just ignore it. Some block port 25. But is there a better way?
I’m going to propose the following:
- A random check of 1 out of every 100 emails sent through an ISP’s servers, or via port 25 (for ISP’s who allow 3rd party mail servers) get checked by a spam filter (such as SpamAssassin).
- If a user gets flagged, the user enters a “gray list”. In which their emails are checked at a lower interval (1 out of 25) for the next several days.
- If more than 10% get flagged (a rather large margin for today’s Spam Filters). That account should be suspended and investigated by the ISP before being re-enabled.
The vast majority of the above can be automated. But how would this cut down on spam?
Explanation
The vast majority of users send less than 100 emails a day. So the percentage of extra CPU required would be relatively minimal for each legitimate user an ISP has (only 1/100 of outgoing email would be scanned). Odds are the user will have 1 email scanned every 3-7 days (assuming they send between 15-20 emails a day) . For a spammer, or a computer infected with a Trojan, this computer will be sending large sums of spam (perhaps hundreds an hour). It will be rather likely to have one fall into the group tested by the spam filter. Then when it falls into the gray list, it will become rather obvious if it was a fluke (emailing a spouse about Viagra), or a spammer. Spammers need to send bulk amounts of mail to be profitable, since not many who get it actually click and buy something.
Why would an ISP want to bother?
A spammer not only can put a large burden on a mail server (read: cost), but cause an ISP to be blacklisted. This is a negative thing for any ISP because it reduces the quality of service for legitimate users, and could cause customers to feel they can get better service elsewhere. The best way to avoid being blacklisted is to keep your mail servers clean.
Wouldn’t this violate privacy policies?
Not likely. Many ISP’s already scan incoming email for spam and viruses. This is simply applying it in the reverse. There’s likely no additional privacy concerns by doing it this way.
Couldn’t this prevent many virus outbreaks?
Yes, it could be done to prevent viruses, simply by doing the above with a virus scanner.
Could this be done without a “gray list” to make it easier to implement?
Yes, in theory it could. You can just flag an account so an admin is aware. Or suspend right away. Suspending right away (on 1 catch) may cause more false positives than you would want, so I’d advise against it. I’d opt towards flagging an account or perhaps notifying an admin by email. If someone is a real spammer, they will be part of the random sampling a dozen or so times rather quickly. So it will be rather obvious. A “gray list” is more programming, but makes the system more automatic and tolerant. Providing a better experience for end users, with less work for admin’s in the long run.
Where did 1 out of 100 come from?
It’s somewhat arbitrary, but should prove effective. I’m sure some analysis could come up with an even better number. The goal is to prevent spam with minimal CPU. Odds are a spammer won’t send 1 email a day. So they will send it in volume (since the more they send, the higher the chances a consumer will bite). Hopefully more often than note, 1 will fall into the filter. You can cut that in half (1 out of 50) to double your chances. At the expense of system resources.
Wouldn’t this just make email slower?
Not really. You can send the email before you scan it. So this doesn’t slow outbound email. It’s just taking a random sampling at an interval, and reacting based on the analysis. Even if the filter goes off, the mail should be sent (it could be a false positive). Only when the user is flagged as a spammer should the account be unable to send email. This results in minimal disruption of service. For a spammer this should happen relatively quick. scanning 1% of outgoing email shouldn’t be to substantial. Assuming you keep an eye on your mail server anyway, this should only speed up the detection of a spammer using it. If you go to a 1:50 ratio of scanning, you’ll only improve your odds and speed in catching spammers.
Has anyone implemented this? Is there a tutorial?
To the best of my knowledge, nobody has done this yet, at least based on my theories. If you have done this, and would like to contribute some code, information, wisdom, or just mention who did it, let me know.
Why not just scan all outgoing email?
It’s just not practical for performance/resource reasons. Nor is it really necessary, since spammers need to send in bulk.
Couldn’t spammers work around this?
Well, they can space out when they send out mail, say batches of 50, but they still fall trap to perhaps being 1:100 and being scanned. They could send less, but that would be costly. They need to send in bulk so they can get as many eyes looking at their offers as possible. So for them, just sending less isn’t good business. This would hit them where it hurts. By making their business model ineffective. If they can’t send the mail, they can’t profit.
Doesn’t this protect others, rather than myself?
Yes, and no. We are a community, and communities do look out for each other. If everyone did this, the load on incoming mail servers would be substantially less. As said before, by catching your own spammers, you prevent being blacklisted by the many blacklists out there. That has a direct benefit to your business.
What about bounced email?
Those should be scanned as well. Simply because a spammer can bounce their spam off of your mail servers to get around blacklists. If I email invalid@goodisp.com, with a spoofed “From:” header, they will likely “bounce” that email to my recipient (who I put in my “from:” tag), quoting the message (my spam). By scanning these as well (1 out of 100), you can effectively cut down on this abuse by your leeching spammers.
The bottom line
By using the above method of scanning outgoing email, you can effectively prevent spammers from profiting off of your mail servers. Spammers need to send in bulk. The more they send, the easier it will be to catch them. This is an easy way for an ISP, webhost or mail provider to cripple the spammers business without harming legitimate email users.
I noticed this on Google this evening.