On Thu, Feb 03, 2005 at 02:49:12PM -0600, David Hoffman wrote: > I looked for any discussion lists about greylisting and haven't found > any, so I thought I might try asking here. > > I'm considering adding greylisting to my postfix configuration, and > some of the articles I have been reading about greylisting show that > there can be any of several situations in which greylisting would not > be a viable solution. > > In particular they mention issues with how some MTAs break something > in the RFC that makes greylisting work, and how receiving mail from a > site which uses multiple relay hosts (each with a different address) > can also cause mail to not be delivered. > > So I thought I would ask on the list to see if anyone has done much > with greylisting and found it to be good or bad. > > I do also use the DNSBL lists, and some of my accounts also use TMDA. > What I am hoping for is that with greylisting, I can further reduce > the amount of spam mail traffic. It is inadvisable for anyone using email in a professional capacity to use any form of TMDA (whitelisting/greylisting). Why? Simply put, it makes possible clients, customers, contacts and potentially valuable network points (people) have to jump thru hoops to make an initial contact with you. Some won't bother, you'll lose. Here is a collection of responses to TMDA which I have seen on the web: ##################################################################### You have sent some one at this domain a whitelisting email. All such emails are blocked at this domain. If you want to communicate with this domain via email, Do not use white listing software or html email. Please do not respond to this email. Please add this domain to your blacklists. We do not want to communicate with you until you remove your whitelisting requirement. Whitelisting is a victory for spammers. ###################################################################### Why White listing is harmful to the good users of the internet and not harmful to spammers: Please read the following discussions on the problems with whitelisting: Quote: http://gnosis.cx/publish/programming/filtering-spam.html "Although I have not used any of these tools more than experimentally myself, I would expect whitelist/verification filters to be very nearly 100% effective in blocking spam messages. It is conceivable that spammers will start adding challenge responses to their systems, but this could be countered by making challenges slightly more sophisticated (e.g. requiring small human modification to a code). Spammers who respond, moreover, make themselves more easily traceable for people seeking legal remedies against them. The problem with whitelist/verification filters is the extra burden they place on legitimate senders. Inasmuch as some correspondents may fail to respond to challenges--for any reason--this makes for a type of false positive. In the best case, a slight extra effort is required for legitimate senders. But senders who have unreliable ISPs, picky firewalls, multiple email addresses, non-native understanding of English (or whatever language the challenge is written in), or who simply overlook or cannot be bothered with challenges, may not have their legitimate messages delivered. Moreover, sometimes legitimate "correspondents" are not people at all, but automated response systems with no capability of challenge response. Whitelist/verification filters are likely to require extra efforts to deal with mailing-list signups, online purchases, website registrations, and other "robot correspondences."" ENDQUOTE QUOTE: http://tardigrade.net/tmda.html "TMDA will prevent you from getting a wide variety of real mail. Some varieties prevent the disabled from completing the verification process. TMDA will prevent you from registering at many web sites, buying software when they email you the registration key, or receiving receipts and shipping notices. I'm far from the only real human who absolutely refuses to jump through hoops such as this. Ah, you say, you can periodically check the rejected mail to make sure you aren't missing anything good! At which point, why bother with it at all? Use a simple set of mail client filters and you're better off--same number of spam subject lines to scan for false positives, and you'll never confuse or irritate any real people. TMDA is guaranteed to keep you off of a lot of mailing lists, and you may never know why, because no one can tell you without jumping through hoops. The list server won't be able to send you a confirmation request. If you do manage to subscribe to a list somehow, it's downright rude to send such messages to the people who post to the list, and just as bad to direct them to the listowner. You've already explicitly agreed to accept list mail by subscribing at all. As a listowner, I'd never allow a member to punish contributors that way. TMDA has come up on several lists for listowners recently, and the opinion has been unanimous against the technique. The 'jump through hoops' message sent out to legitimate correspondents is even more annoying than spam is. Dealing with incoming spam directly is a nuisance, but missing out on real mail can be the pits. Prospective employers aren't going to jump through hoops to send you a job offer. If your great-uncle gets confused about the process, you'll miss the invitation to a family reunion. What if your out of town wife has to send you an urgent message, but can only do it from a borrowed account at the airport before she catches a plane? Just say NO to TMDA." ENDQUOTE QUOTE: http://216.239.53.100/search?q=cache:HJl4Fo1tdhYC:208.171.236.113/cpunx-news/cpunx-news20020311/0009.html++%22whitelist%22+email+bad&hl=en&lr=lang_en&ie=UTF-8 So, I got to say, I really, really, really hate this auto-reporting white-list challenging crap. It's goddxxxed rude to your absolutely legitimate correspondents. The problem is that any auto-reply or challenge makes me jump through some kind of hoop just because *your* spam filters are not smart enough to tell the difference between my worthwhile mail and some UCE. Admittedly, the kind of language recognition that would be able decisively and without fail to detect spam is astronomically hard. But that still doesn't make it right for *me* to have to pay the price for their failure or indecision. It's asinine of you to put the time cost that your spam incurs into *my* ledger. I have my own dxxx spam to deal with, and I don't want to have to pay the price in time and effort for dealing with _your_ spam, too. *I* didn't write your crummy bubble-headed coarse-grained filters, did I? It'd be much, much better for you just to flag suspicious messages and put them in a slops bucket folder that gets checked and cleared out once a week. Sure, it's 30 seconds extra work for you to scan the folder, find my diamond, and whitelist me, but at least *you* are the one making the effort to keep your own damn inbox free of spam. Spammers are Bad because they abuse the time, attention, and digital sources of others without permission. Everyone who sends out "My filter thinks you're spam so jump through this hoop" messages, are doing the same damn thing. They are a tinhorn Sanford Wallaces of the 21st Century. I'm sick of doing the spam-fighting work for lazy basxxxdos who consider themselves quite smart for sending out autoreplies. Get over yourselves! You're not that goddamned important. ~Mr. Bad P.S. I apologize to anyone who's already seen this rant in one form or the other. I've sent it out like 6 times this week. Half the time I get back messages that completely miss the point, saying, "But spam is really bad!" No shxx, sherlock_at_holmes.com. So is being a rude xxxhole to everyone who's writing email to you." ENDQUOTE QUOTE:http://lists.debian.org/debian-devel/2002/debian-devel-200212/msg00136.html "Here's my problem with such tricks: They take the personal (and best addressed as a personally-managed) problem of whitelist generation and offload it to a class of people who neither particularly care, nor are skilled at, executing it. As is clear here, the tactic is spectacularly ill-suited to mass communications, mailing lists in particular. If I'm posting mail to a list, WTF should I care what Joe Bumpkiss, or Gerrit Pape, wants to do with my email? If s/he signed up for the list, the presumption is that s/he wants to receive the mail. Ordinarially[1] I use a set of procmail recipies which filter mail on a number of criteria. These include heursitics to detect list mail, spamassassin, and a set of white and black lists. With my mailer, it's trivial to select a message, or a list of messages, and add the sender to either my white or black list. Takes a fraction of a second. Only happens once (and generally only for mail directed to me -- list mail doesn't need this hoop).[2] Best of all, my system never reveals itself to the sender at all. Which is as it should be. I roundfile any "prove yourself" requests I receive, and blacklist the sender. Peace. -------------------- Notes: 1. System failures mean I'm on a fallback mail system w/o my procmail support. Two days of filtering by hand... I'm going to dig through backups to get 'em back in place RSN. 2. The system is based on the Debian spamfilter package, Lars Wizenius's procmal recipies. Spamassassin support is simply added as another rule. I've added a small script to add an address to a b/w list. -- Karsten M. Self <kmself@xxxxxxxxxxxxx> http://kmself.home.netcom.com/ What Part of "Gestalt" don't you understand? Geek for hire: http://kmself.home.netcom.com/resume.html ENDQUOTE READ This article at slashdot: http://slashdot.org/articles/03/02/04/161252.shtml?tid=111 Try this scenario: N is the number of people using spamarrest (A TMDA system) K is the number of people using email on the internet. How much is N raised to K-th power? Thats how many of these spamarrest emails will start flying around the net. How much time will it waste for people? more than spam wastes. Whitelisting will not affect spam on the Internet. Get a spam tool that doesn't hurt non-spammers. Things that make sure white listing tools will never work: 1.) Dynamically generated email addresses - Like all of mine. Spamarrest will require me to "justify" all of them. Do you think I'm going to bother to do that ? Nope, Not a chance and neither will any of the other people who have any length of experience. 2.) Faked email headers : Email headers can be forged, so email can be made to look like it comes from people you have already whitelisted. This makes people who cooperate with spamarrest type tools prime targets to be used as forged email sources. They will then be blacklisted by all the RBLS services which will cut them and their ISP off. Severe damage done to them. They can even be sued by their ISP for getting the ISP blocked unless they can prove they didn't do it. proving they didn't do it requires a sophisticated level of Internet protocol and how-to knowledge beyond most people. 3.) Faked email envelope information : Email envelope information can be forged, so email can be made to look like it comes from people you have already whitelisted. Note - email envelope information is not the same as the email header information. Even if the verification system becomes more intelligent and uses some of the tools now being provided to requires a human reader to interpret images of text to do the verification, the spammers can fake that too. Its actually very easy to interpret those text images with existing open source software. There are more reasons but the above info should be enough reasons for you to turn off your whitelisting tools. Quote:http://gnosis.cx/publish/programming/filtering-spam.html: [Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index][Thread Index][Top&Search][Original] Re: jump-through-my-hoops autoresponders * From: "David W. Tamkin" <dattier@xxxxxxxxx> * To: procmail-users@xxxxxxxxxxxx * Date: Thu, 13 Dec 2001 15:08:57 -0600 * Message-id: <00b701c1841a$db353d00$fb080142@xxxxxxxxxxxxxxxxxxxxxxxxxxx> I started this post Wednesday evening but put it aside. In the interim John Conover sent private mail to Sean, Mark from asarian-host.net, and me detailing how his whitelisting system does what it can to avoid acting on mailing list distributions, NDNs, and comparable "prove yourself" messages from other whitelisters. (The last, of course, depends on the other whitelisters' nastygrams' conforming to some expectation.) That's good, but I didn't intend to say anything more about the damage these systems do when the user blows off whitelisting a mailing list, so I'm picking up where I left off. Previously, John had explained on-list, | The way it works is that the mail system on a gateway maintains a | whitelist, (where the whitelist is only available to the system | folks.) So you wrote it for gateways, not for individual users? If it were for keeping out non-business email that has no place entering a corporate system, that would be a whole other story. But per your description an employee's non-work acquaintance who is sending personal email can just jump as instructed and get whitelisted, so it doesn't accomplish that. Meanwhile, unexpected business-related mail gets blocked and refused. Clients and suppliers alike must adore having their communication rejected because the admins don't bother whitelisting them or because they wrote from a different server at their own company. I can't think of anything less professional than a "you have to prove you aren't a spammer" autoresponse to a customer or an established business contact whose email address somehow wasn't already on the whitelist. It sounds like a very good way to run a firm into the ground. | I really don't understand what all the emotion is about ... Guess what, John: people aren't computers. We have emotions. We notice insults and snootiness, and even when we don't retaliate we certainly don't keep coming back for more. You've obviously never been on the receiving end of it, never answered someone's plea for help or a question mailed privately to you only to be punished with the equivalent of "How dare you think you're worthy of emailing ***ME***, insignificant speck! If you even dream that I'd read it, you'd better jump through my hoops first!" I've been subjected to that crap four times (including once through this list) in the past few years. (Anyone else remember how Eli the Bearded's .sig used to say, "It is not my responsibility to prove to you that my mail is not spam"?) Beyond being insulting, it is foolish, as Sean explained, because the autoresponse can achieve one of only four results: [1] It will be directed to an invalid address and thus accomplish nothing, because [1a] the triggering message was spam with a forged return or reply address, or [1b] the triggering message was legitimate but a broken gateway hosed its return address, so the sender will never get the autoresponse and never, even if willing, follow its instructions. (However, its reply address was intact, so it was possible to answer it if the whitelister ever read it; or maybe it required no response but still should have been read.) [2] It will be additional junk mail heaped upon an innocent party whose address was forged as the return or reply address of spam and thus will be spam itself. [3] It will insult someone who was sending legitimate mail, often someone to whom the whitelister had written first asking for help or for answers; that's why [as you have acknowledged] most people do not bother groveling for admission to the holy whitelist and valid email goes unread. or [4] It will validate the whitelister's address to a spammer who uses a monitored return or reply address. In case 1a, the setup just uses bandwidth for no result; in cases 1b, 2, 3, and 4, it does harm. It does not get the user off spammers' lists; it saves the user from using the MUA's delete function but the price is missing a lot of legitimate mail and becoming a spammer him/herself. That's not to say the idea couldn't be made slightly less bad ("improved" is too strong a word): [a] The software should automatically whitelist (1) addresses to which the user writes, (2) subjects of the user's outgoing email, and (3) subjects of the user's posts to netnews, and then most replies to mail or posts from the user would come through. (Whitelisted subjects, of course, should expire after a while.) But none I've heard of, not even yours, support even a manually maintained whitelist of subjects, and few, not even yours, automatically whitelist addressees of the user's outgoing mail. [b] The tone of the canned message should be sympathetic for having to ask the sender to go to such lengths because of everybody's struggle with the spam problem, but all those I've seen are presumptive, confrontational, accusatory, and haughty. They are phrased with the deep conviction that the sender of the triggering message can be nothing except a spammer, but as an afterthought the whitelister, being a gracious and merciful superior being, offers the unworthy sender one final and highly undeserved chance to atone for the grave sin of sending email. Maybe the grand majority of these autoresponses are triggered by the arrival of spam, but 100% of those that reach human eyes are read by non-spammers, most of whom are responding to email or posts sent by the whitelister. The correct attitude should be, "Your email reached my site, but my email filtering routines took it for spam and returned it to you; if you are reading this, clearly they erred and they must be adjusted. Please forgive me and send it back by replying to this message. Thereafter the filters will allow mail from your address through without trouble. Thank you, and I'm very sorry about this. Usually this rejection message goes to spammers, but unfortunately the system sometimes mistakes good mail for spam." [Of course "sometimes" is a lie for "almost always."] It should be phrased under the assumption that the message was *not* spam; it should not boast about one's whitelisting setup nor one's need to keep out the riff-raff; it should present the remailing instructions as a way of getting past overzealous spam filters rather than as a way of applying for admission to a whitelist of tolerated senders. Do you follow? It should say that the whitelister is wrong for treating the message as spam, not that the sender is wrong for sending email without having first run the gauntlet to gain membership in the whitelister's tiny circle of special friends. Does your software come with a sample text or a default text that is phrased for the non-spammers who will read it rather than for the spammers who will not, and does it come with clear instructions that any edits to the text should take that into account? I'm guessing that its default text is more like, "We accept mail only from people on a list of pre-approved senders. To get onto our list, reply to this message." [c] Finally confirmation should be easy, such as just replying to the autoresponse as you said, and should not require revising it to include magic words or the like. But then that runs afoul of an autoresponder at the sender's end. I can think of some ways around that, but they're not so good. (For example, include two codewords in the confirmation instructions and require that a confirmer delete the first but leave the second one intact in the remailing; autoresponders will return either none of the body, all of it, or a selection truncated from the beginning, so they'll fail to confirm. But that's more involved than just sending it again, so that's a drawback. Best solution: drop the whole misbegotten notion.) Even so, my suggestions will help only with #1b and #2. And still, despite theories of how it could be improved, in practice it's just plain a bad idea used by rude people with entitlement issues who order others to screen their email for them and don't care what mail gets lost or bridges get burned in the process. You are harming your reputation, John, by promoting a product that implements it. There is only one decent way to find out whether a piece of mail is spam, and that is to look at its content. False positives are worse than false negatives. And there is only decent one way I can think of to use a whitelist: 1. Automatically add addressees of your outgoing mail and subjects of your outgoing mail and netnews posts. 2. If an incoming message fails to match the whitelist, divert it to a low-priority folder and check its content with your own eyes before you do any conclusion-jumping or demand any hoop-jumping. 3. Do *not* autorespond. If a valid message lands in the low-priority folder, quietly update your whitelist. It isn't the other person's job to fix it for you. END QUOTE _______________________________________________ QUOTE: http://www.nclug.org/pipermail/nclug/2002-February/003143.html: last summer i helped organize my 20th highschool reunion. whitelists would have _way_ added to the suckiness of that job. > > They that can give up essential liberty to > obtain a little temporary saftey deserve > neither liberty nor safety. > -- Benjamin Franklin, 1759 END QUOTE You changed your behavior to stop the spammers - and it won't stop them! You lose. I can write a one line Perl script to to respond to a spam arrest request for email verification automatically. If I can do it, so can the spammers. (Note: and there is evidence that they already have.) Some whitelisting software makes unwarranted assumptions about many things. First it assumes that everybody reads their email from inside a browser or from inside a Microsoft based email reader that will activate a browser window when a user clicks on a URL inside the email. Most experienced email users don't use a browser to read email in. Why? It's too dangerous. HTML email as inherently dangerous. It not only allows spammers to verify that your email address is valid just because you read the email it can allow javascript programs or other plug-ins to be executed by an email. This is very risky hence it is not allowed at many sites. Whitelisting doesn't hurt the spammers but it does cause those who attempt to communicate with a whitelist user to be damaged by forcing them to take extra steps and do extra work to get the communication to happen. Since many people participate in email lists where they may communicate in a person to person email (not a list email) with many many people just a few times the added work whitelisting will require to communicate with each one will kill the email lists. -- Linux/Open Source: Your infrastructure belongs to you, free, forever. Idealism: "Realism applied over a longer time period" http://www.scaled.com/projects/tierone/ http://kinz.org http://www.fedoratracker.org http://www.fedorafaq.org http://www.fedoranews.org Jeff Kinz, Emergent Research, Hudson, MA.
