Greylisting -- In development
You can read about this technique everywhere (You can even read the original paper), but we'll look at the broader picture a bit here.
The Greylisting concept entered discussion in anti-spam circles around five years ago and immediately received a lot of attention, mostly negative at the time. The term places the system between white-listing (a list of sources you will accept mail from) and black-listing (a list of sources you won't accept mail from). One can think of it as a sort of undecided list.
Greylisting works (in its original implementation) by initially rejecting all mail in SMTP using a transient error message. According to the SMTP specification, mail rejected with a transient error must be queued and sent again later. The recommendation is to queue mail for at least 24 hours, but in practice the second attempt takes place quite promptly. A Greylisting server would not apply the transient error rule to the same message when it is resent. On retry it should either accept mail or permanently reject it.
At first glance one might wonder what the point is. How does this separate spam from non-spam? The answer is "scale". Spam, by its nature, is bulk e-mail. These days spam campaigns regularly have millions of recipients. LightningMail advertises that they will send spam to 2 billion recipients for just $25 USD, and there are plenty of larger campaigns out there.
At this sort of scale it is simply impractical to actually store that much mail. Even keeping track of how recipient transport agents responded to the addresses requires a vast amount of memory. For such systems, a transient error is the same as a permanent error. As a result, rejecting mail with a transient error very effectively separates legitimate senders (using SMTP compliant transport agents) from unwanted senders (using real-time message generating software talking with SMTP).
Our feelings at the time were that spammers would just adapt (as they always have) to send multiple messages to each recipient instead, resulting in increased spam volumes that defeated the system. Surprisingly, this doesn't seem to have happened yet. However now that we have very effective real-time black lists (which did not exist five years ago) it may not matter. Just delaying the spam campaign long enough for these black-lists to be updated will mean that by the time the second message comes in the sender has already been identified as a spam source.
We are now working on a grey-listing system that is both reliable and effective, without putting undue strain on peers. We have also avoided this technique because it shifts the burden on to legitimate mail transport agents. However, now that the vast majority of all traffic is spam it is increasingly important to use all available methods to control it. Simply allowing spam to be transmitted at all is wasting someone's resources.
We believe that we can deliver a system that punishes innocent peers as little as possible by using heuristic analysis to determine whether or not to apply grey-listing at all. This would make use of, for example, softfail SPF, failed domainkeys, low SpamAssassin scores, low Bayes scores, dynamic-ip-blacklisting and so on. While one might not rely on these methods to reject mail outright, delaying delivery by a small amount of time (at the administrator's discretion) would be a small cost when spam reduction could be significant.
Greylisting will be available in Mailtraq soon, and will be a free upgrade with valid Upgrade Protection
