...making Linux just a little more fun!
Joey Prestia [joey at linuxamd.com]
Hi all,
I am wanting to gather some information about using DNSBL on mail servers. I have been reading the information on most of the more popular used blacklists like Spamcop and Spamhaus. Now I have come up with all kinds of questions on the subject.
I would like to hear from any mail server administrators of their experiences with these methods of rejecting spam at the "gate". It seems apparent that one must gage what type of spam and what type of lists to use very carefully because of the possibility of refusing valid mail?
Is the implementation of using a DNSBL definitely something mail server administrators should consider?
Is it common practice to use spamassassin and DNSBL together to reduce bombardment of spam?
Although I have been using spamassassin for some time and see that it does a very good job of filtering and correctly labeling mail. Also the majority seems it could be prevented altogether by implementing the correct DNSBL or DNSBL's at the mail server level as I can see by spamassassin headers.
One thing I have heard is that it is not a good practice to put into effect something like this because many bigger institutions can and periodically do get put on blacklists, through no fault of their own. One example I have seen: https://www.stanford.edu/services/email/antispam/blacklist.html is this an accurate representation of some of the possible effects of this being put into practice?
Any recommendations as to suggested best practices in using these measures?
Thanks,
-- Joey Prestia L. G. Mirror Coordinator https://linuxamd.com Main Site https://linuxgazette.net
Faber J. Fedor [faber at linuxnj.com]
On 19/09/08 19:09 -0700, Joey Prestia wrote:
> Hi all,
Hi,
> Is the implementation of using a DNSBL definitely something mail server > administrators should consider?
I think so. The only downside of that is when the DNSBL "goes out of business" which slows down email delivery but is otherwise harmless, but overall, I suggest using it.
However, my favorite was of dealing with spam is to reject email that doesn't follow the RFCs. That's why I like postfix; they make it very simple to reject emails that don't follow proper procedures (https://jimsun.linxnet.com/misc/postfix-anti-UCE.txt).
> Is it common practice to use spamassassin and DNSBL together to reduce > bombardment of spam?
When I was running my own Postfix server, I got very little spam. I
switched over to a hostgator account and my only choice for spam
filtering is SpamAssassin. It's not as good as using Postfix's anti-UCE
techniques + SA + procmail, IMO. I now get several spams (> 10) a day.
> One thing I have heard is that it is not a good practice to put into > effect something like this because many bigger institutions can and > periodically do get put on blacklists, through no fault of their own. > One example I have seen: > https://www.stanford.edu/services/email/antispam/blacklist.html is this > an accurate representation of some of the possible effects of this being > put into practice?
Yes, it happens, but over the years I've come to appreciate Rick Moen's no-nonsense, hard-@$$^H^H^Hnosed approach: if Stanford gets blocked due to a blacklisting, you'll hear about it and can whitelist them.
-- Regards, Faber Fedor President Linux New Jersey, Inc. 908-320-0357 800-706-0701
Kapil Hari Paranjape [kapil at imsc.res.in]
Hello,
On Fri, 19 Sep 2008, Joey Prestia wrote:
> Is it common practice to use spamassassin and DNSBL together to reduce > bombardment of spam?
At least this is "common" at our IMSc mail server. This means that we use DSNBL's as part of our spam scoring strategy.
I have found that rejecting mail based only on DNSBL's is fraught with problems like the one you described. The situation is worse in India since many ISP's completely ignore warnings about spammers operating within their IP address ranges; this means that the DNSBL's tend to classify the complete range as spammers.
Since we "herd cats" here ( ) we want to avoid being lacerated for
false positives.[1] Hence our strategy is to check for RFC compliance
(we use exim4 for this). We also verify that the sender (Envelope From)
of a mail can be reached (bounce check). Finally, we have SPF records
and we do not accept mail from outside our LAN that has a sender
(Envelope From) within our domain. These are the only forms of
SMTP-time rejection that we do.[2]
We then score the mail using spamassassin and let our users filter mail (or not) as they want (there are help pages on how procmail can be used to separate mail based on spamassassin's scores).[3]
I admit that this is not the best strategy for a site where a significant chunk of bandwidth cost is due to mail.
I have heard that allowing users some control on SMTP transactions based on their own scoring can be more effective (greylistd for example). We haven't had the time to test it here.
Regards,
Kapil.
[1] False positives is the term for mail that is genuine and gets classified as spam.
[2] Even with this minimal setup I see that about one-third of all SMTP connections to our mail exchangers are rejected everyday.
[3] Statistics based on spamassassin's classification of my inbox (which is post the SMTP rejections in [2]) seems to indicate that about two-thirds of the messages getting through are still spam. --
Michael Makuch [linuxgazette at makuch.org]
I have been running sendmail for my own personal and business email for the past eight years or so. And I have been using DNSBLs for most of that time. I think DNSBL is a great tool to fight spam. I do not use any type of spam filtering that requires the user to sift through junk/bulk folders as IMHO that completely misses the point. I have encountered two issues with using DNSBL that were easily addressed.
The two issues I have encountered are 1) blacklists sometimes list a whole class C when they should only be listing specific IPs and 2) I recently ran into a problem with the latency involved in using DNSBL.
Occasionally the ISP of a friend or business associate will become blacklisted and that person can no longer send email to me, they bounce back. Well when this happens it is invariably someone who knows another way to contact me and does so. At that point I check my maillog and whitelist the ISP. Solved.
Recently one particular ISPs mail server began timing out while communicating with my sendmail. I narrowed the problem down do a DNSBL that had gone out of business, removed it from my config and all was fine again.
Initially I used a long list of DNSBLs including spamhaus, abuseat, njabl, blitzed, dsbl, ordb, JAMMConsulting and uceprotect. But over time I have shortened that list and I currently only use zen.spamhaus.org.
I recommend DNSBL. I wish that many more ISPs would use DNSBLs as I think they would work even better.
Mike
Joey Prestia [joey at linuxamd.com]
Michael Makuch wrote:
> Initially I used a long list of DNSBLs including spamhaus, abuseat, > njabl, blitzed, dsbl, > ordb, JAMMConsulting and uceprotect. But over time I have shortened that > list and I > currently only use zen.spamhaus.org. > > I recommend DNSBL. I wish that many more ISPs would use DNSBLs as I > think they would work even better. >
So using multiple DNSBL's together is relatively safe. I have heard that using zen.spamhaus.org is not recommended for users using SMTP Authentication on their server as it might block the roaming users from authenticating.
Although I would think that perhaps the priority of the lines in the configuration of sendmail could be arranged so as to allow users that authenticate to bypass any ruleset?
Actually I would think that authenticated users could bypass such rules regardless just as they do in order to relay?
Best,
-- Joey Prestia L. G. Mirror Coordinator https://linuxamd.com Main Site https://linuxgazette.net
Michael Makuch [mike8 at makuch.org]
Joey Prestia wrote:
> Michael Makuch wrote: > >> Initially I used a long list of DNSBLs including spamhaus, abuseat, >> njabl, blitzed, dsbl, >> ordb, JAMMConsulting and uceprotect. But over time I have shortened that >> list and I >> currently only use zen.spamhaus.org. >> I recommend DNSBL. I wish that many more ISPs would use DNSBLs as I >> think they would work even better. >> > > So using multiple DNSBL's together is relatively safe. I have heard that > using zen.spamhaus.org is not recommended for users using SMTP > Authentication on their server as it might block the roaming users from > authenticating. > Although I would think that perhaps the priority of the lines in the > configuration of sendmail could be arranged so as to allow users that > authenticate to bypass any ruleset? > Actually I would think that authenticated users could bypass such rules > regardless just as they do in order to relay? > Best
I'm not sure if sendmail can be configured that way or not, I don't authenticate and it's been a long time since I opened up the Bat book (viva m4). Yes I am sure that a larger ISP would run into problems. I just mean that, if more people were using DNSBLs, then the lists would be better maintained and there would be fewer problems, etc. Catch 22 I guess.
Rick Moen [rick at linuxmafia.com]
Quoting Joey Prestia (joey@linuxamd.com):
> I am wanting to gather some information about using DNSBL on mail > servers. I have been reading the information on most of the more popular > used blacklists like Spamcop and Spamhaus. Now I have come up with all > kinds of questions on the subject. > > I would like to hear from any mail server administrators of their > experiences with these methods of rejecting spam at the "gate".
Your question seems to presuppose that DNSBLs are a "method". They're not. They're a source of information that can be used in a variety of ways.
Information has value to the extent that you have confidence in its accuracy and relevance -- which includes deciding whether or not you (as MTA admin) think their listing policies are reasonable. Experience suggests that that's a question one must assess individually for each DNSBL. Further, DNSBLs have been known to be good for a while, then not so good, then eventually may become counterproductive, e.g., because they've ceased to be maintained.
It might seem counterintuitive, but one of the best things that some past DNSBLs have done for the public was to -- after reaching a decision to no longer maintain the database -- configure their DNSBL nameservers to return a blacklist recommendation on every IP queried. Why? Because that's really the only effective way to notify MTA admins around the world that they need to cease consulting the (now-obsolete) DNSBL.
Anyway, as I said, DNSBLs are not a "method" but rather a source of information. Many of us MTA admins find it most fruitful to feed that information through a spamicity-rating service locally, e.g., raising or lowering the SpamAssassin score with weighting set for each DNSBL according to the degree to which we think each one should be, on average, taken seriously.
> It seems apparent that one must gage what type of spam and what type > of lists to use very carefully because of the possibility of refusing > valid mail?
Of course. This survey might be one good starting point: https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists
> Is the implementation of using a DNSBL definitely something mail > server administrators should consider?
Yes -- for appropriate values of "implementation".
> Is it common practice to use spamassassin and DNSBL together to reduce > bombardment of spam?
Sure.
> Although I have been using spamassassin for some time and see that it > does a very good job of filtering and correctly labeling mail.
SpamAssassin is a ruleset and a clearinghouse infrastructure for receiving and processing information from arbitrary sources about a mail stream's estimated spamicity.
> One thing I have heard is that it is not a good practice to put into > effect something like this because many bigger institutions can and > periodically do get put on blacklists, through no fault of their own.
Foregoing sounds like some combination of
1. non-sequitur reasoning, and 2. special pleading (https://en.wikipedia.org/wiki/Special_pleading)
Early in the spam wars, special pleading was even more common than it is today. E.g., you as MTA admin would mention that you were likely to join the rest of the sysadmin world in refusing mail from all known open-relay SMTP sites, because there was simply no excuse for operating such a site in the cutting-edge early-1990s world you then inhabited -- and immediately some nitwit from a local university would be on your case:
"But, but, but... we have to operate open relays because our users insist on being able to relay outbound mail on port 25 from arbitrary IPs, and we university sysadmins lack the political power to order them not to, and it's simply unfair to hold the entire university responsible for what a few bad apples do."
(I swear I heard exactly that sort of thing, back in the day, many times.)
What I'm driving at is: Some bigger institutions get put on blacklists because, by and large, they got caught behaving like anti-social screwups, i.e., for very good reason. They can get off those blacklists by following the delisting policies, which by and large loosely approximate "Stop screwing up".
Of course, there are DNSBLs with bad/ineffective/arbitrary/onerous delisting policies -- but far fewer than asserted by critics who include... surprise... habitual screwups who are following the other bit of logic that accompanies special pleading: blame everyone and anyone but one's self.
> One example I have seen: > https://www.stanford.edu/services/email/antispam/blacklist.html is this > an accurate representation of some of the possible effects of this > being put into practice?
The page doesn't give enough technical information about the "forwarding to AOL" example to assess it. I'm not certain offhand how the hypothetically forwarded spam received at Stanford and then forwarded manually, or via a .forward file, could be expected to look (the SMTP headers) as received at aol.com. Therefore, I'm not certain whether SpamCom can be rationally expected to be able to distinguish between Stanford being a genuine spam-source on the one hand, and it being an innocent recipient of spam that one of its users then forwarded to his/her AOL account, on the other. My guess, having not seen the headers of such a forward, is that the Stanford page makes a good point, and that SpamCop is vulnerable to submission of bad information.
As it happens, I don't personally use SpamCop's DNSBL information, because I don't particularly trust it.
> Any recommendations as to suggested best practices in using these > measures?
Yes. Read DNSBL listing and delisting policies carefully. Use the resulting DNSBL information with weighting according to your confidence and agreement with their policies and according to your understanding of their competence and good faith. Take critics' bitching cum granum salis, but also be aware that sometimes they're right even though they're complaining. ;->
Joey Prestia [joey at linuxamd.com]
I would like to thank everyone for their replies on my questions about using DNSBL's. I've gotten a lot of good information from everyone that replied. As well as gained additional resources for more information.
Thanks,
-- Joey Prestia L. G. Mirror Coordinator https://linuxamd.com Main Site https://linuxgazette.net
René Pfeiffer [lynx at luchs.at]
On Sep 21, 2008 at 0756 -0700, Joey Prestia appeared and said:
> Michael Makuch wrote: > > > Initially I used a long list of DNSBLs including spamhaus, abuseat, > > njabl, blitzed, dsbl, > > ordb, JAMMConsulting and uceprotect. But over time I have shortened that > > list and I > > currently only use zen.spamhaus.org. > > > > I recommend DNSBL. I wish that many more ISPs would use DNSBLs as I > > think they would work even better. > > So using multiple DNSBL's together is relatively safe. [...]
I am doing this for years now, and it works very well.
> [...] > Although I would think that perhaps the priority of the lines in the > configuration of sendmail could be arranged so as to allow users that > authenticate to bypass any ruleset?
It should be possible. It's easy in Postfix, you just put the SMTP AUTH
check before the DNSBLs and you're done. Another way is to use the
proper ports for SMTP clients (which is not port 25/TCP any bypass
the DNSBLs this way.
Best, René.