At 22:42 10/04/2012 Tuesday, Michael Deutschmann wrote:
>On Sun, 8 Apr 2012, alan wrote:
>> then the block is on ip and thus dosn't reference mailfrom as the issue
>> obviously
>
>So you mean per-IP blacklisting trumps per-sender-address whitelisting?
>
>Does that apply if the per-IP blacklisting is from a DNSBL and not a
>manual entry?
depends on the users preference (some use some dnsbls as just extra points to add to spamscore of content checks)
some dnsbl are a do-not pass (as any whitelisted sender from say pbl is obviously a bot forged spam/virus)
why we advise users if they want to 'whitelist' a sender to do it on more than 1 datapoint (so they whitelist only the sender not the forgeries claiming to be the sender)
so if (here its common for eircomnet to get blacklisted on say spamcop, or was a few years ago before they cleaned up)
so most users whitelisted that ip(s) from spamcop listings
(as they know the ip is legit and want mail from not just x)
whitelisting an address is only logical if whitelisting it from all further content checks
if the address was not previously bogus its unlikely to change, so whitelisting is unneccissary
(ie my virus and spam discussion mailinglist subscriptions are whitelisted from all content checks)
>Sender whitelisting is most commonly used to patch over the occasional
>DNSBL false positive/collateral damage, without standing down the
>defenses entirely. A sender whitelist that did not trump the IP-based
>reputation would be useless for that purpose.
if its to trump a dnsbl entry an ip whitelisting by that user is how its done
(either they believe the ip is clean or not, if they want to refuse mails from other users on that ip they would want to do it regardless of dnsbl reputation, and thus whitelist ***@y and blacklist *@y and whitelist the ip from all dnsbl, to have consistent results)
>Coming back to SPF, regardless of what SPFv1 Pass means, it would be good
>to have a differentition in SPFv3 between a "unbounceable pass" and a
>"bounceable pass".
it would not, if i sent you a mail from totally-***@alandoherty.net with a bounceable pass, it would still clog your dead letter queue as the recipient still dosnt exist for you to return a bounce too.
(in my case as i have per-user spf1 records it would in fact get a record of -all after expansion, but most spf implementers dont save others from their users typo's like we do)
all reasonable senders accept bounces, any that don't should not expect receivers to suppress them artificially
(as blackholed mail is more counterproductive than backscatter)
all reasonable receivers don't (normally) bounce
all unreasonable senders (those that allow their users to forge envelope-sender) will be souces of backscatter when talking to reasonable receivers, and will get dnsbled till they clean up if/when they become sources of backscatter to innocent/forged 3rd parties.
all unreasonable receivers (those that accept then bounce) will generate backscatter themselves (masking the unreasonable sender) and thus equally get dnsbled till they clean up if/when they become sources of backscatter to innocent/forged 3rd parties.
like open relays firstly the unreasonable receivers will have to clean up, in time all senders wanting to be accepted anywhere will also have to cleanup.
(we still have open relays but vanishing few and all on every dnsbl)
eventually abusive recievers will be a thing of the past, a long time later abusive senders will not last long before they get listed due to the recievers that currently mask them being removed
we should not advocate blackholeing or any system that cripples the legitimate flow of the (few but necessary)legitimate external bounces (such as mailbox full, message delayed but still in transit etc[host down/disk full/etc])
just to fix the symptom(backscatter) rather than fixing the problem(otherwise legitimate sender systems that allow their users to and the bots on their pc's to willfully forge)
if ultimately senders cannot forge then like me the only mail a bot(on my pc) could send via my smarthost would result in me getting a flood of NDRs alerting me to the bot using my account)
if the same bot tried direct MX sends no bounces (as all refused due to my pcs IP being in pbl, as its not ip space allowed to direct to MX)
>The distinction would not be used for spam filtering. Setting up a
>conspiracy-compliant spam filter that doesn't blackhole is costly,
blackholing is in no way compliant its just broken
its not costly to engineer systems well/propery (ie if you didn't reject it at the border when you checked it for deliver ability/spammyness/source reputation/etc/etc, then deliver it, if later checks make it look less desirable put it in a spamfolder if you wish but deliver it to the user
> but it is a flat cost. Exempting "bounceable pass" messages doesn't save you
>anything.
>
>Also, even if the spammer has no right to complain of you bouncing back at
>him,
if you reject there is no bounce
>he can still be a jerk when you try to deliver that bounce....
if its sent as a bounce your sending to the forged from 99% not the spammer, why bouncing is abusive
> Some spammers might find it hiliarous to teergrube you, or endlessly 4xx.
no spammers need to the forge the from
>Where it would help with is quota issues.
no it wouldn't
> The more bytes of unbounceable
there is no such thing as (unbounceable mail) if the bounce is legitimately generated
>mail a backup MX is allowed to queue,
if the mailbox is full the backup MX should be 5xx or 4xx ing the mail to sender saying so
not queing mail and lying to the sender claiming its been accepted/delivered
(legit senders want to know asap if they should be phoning the recipient as the mail will likely not get through)
> the bigger the worst-case quota overrun on the primary.
why it shouldn't queue, as 90% of quota overuns are dead/absent/gone/retired/quit/negligent users who lacked the decency to close the account queing in most cases is counterproductive for them and the legitimate senders attempting to mail them
(and the forged-from victims now subject to daily 'your message to x has yet to be delivered' DSNs) why we 5xx and cache.
> So a conspiracy-compliant MX will start 4xxing
>incoming mail long before it comes near to running out of disk.
it should 4xx or 5xx when 4xx'd or 5xx'd from the downstream.
>The MX could add a separate, larger reserve of queue space just for
>bounceable mail -- knowing that if the primary stays down for a month, it
>*can* try to shed the load.
why should it? the sender (if legit) will cache the mail, if non-legit it won't which is a side-benefit
>In the usual case where the primary does come back eventually,
>then it can ETRN
(does any mailserver trust/implement ETRN? i know we do if the domain and connecting ip match (through custom written code), no customer system has ever been found that can avail of it, most customers manually dequeue by issuing 'finger ***@our.mx' from their mailserver (as it is available on widows and unixes and hits our custom listener which only accepts the connect from their ip's and does the same domain/ip matching before dequeue)
>and the waiting mail can be on it's way immediately.
just as easily achieved
>That's the sender's incentive to publish bounceable pass.
so the senders only incentive is that the receiver should silent queue delayed mail, rather than let them queue and let their own policy decide when and how often their own users should be sent 'mail still waiting' DSNs
hardly an incentive?
>But a goldlisting sender can't accept bounces without weakening its
>defense considerably,
err how? a goldlister may have sloppy recievers(bouncers) they need to communicate with, thus they still need to know when their legitimate mail was not delivered due to a misfire of internal content scanner or other)
> so it would *prefer* to get 4xxed
or 5xxd
>than to face blackholing on the slim chance the message is flushed off the bounceable
>queue and cannot be returned due to the goldlisting.
if a goldlister that you describe exists
A their sending system would have to insist the also cannot mail addresses other than those in their 'gold-list'
(obviously as allowing out to addresses not allowed to reply is pointless)
B thus their sending system can easily enumerate that <> from the ip's that are MXs for such goldlist address' and ips that have previously sent from a goldlist address, or are in a goldlist addess's SPF are obviously unlikely to be anything but legitimate bounces (this allowed to <> list could be automatically enumerated and added to every time a new goldlist address is added and refreshed every time one is emailed (as mxs and spfs change)
C bounces can all be allowed to goto data phase (as no legit <> can be to multiple recipients) and thus searched for lines like ***@y mailbox full etc (where address in message body is a goldlist address, or ip in returned headers is a goldlist MX, to catch the occasional edge case multi-stage inbound relays sending from alien IP
but our users thus far that blacklist *@* and whitelist a few have never complained that the whitelist always includes <>
those that do worry about fake bounces opt to turn on batv
>It would be even better to add a third SPF declaration: bouncing is ok if
>and only if bounces can be reliably identified. An obvious way to do this
>is to require both that the bounce be close enough to RFC 3464 & 6522 so
>as to reliably extract the "Original-Recipient" field, that the bounce
>actually contain one (it's optional by RFC 3464) and that SPF would have
>Passed the message had that address been the MAIL FROM.
again spf (sender policy about what ip's my mail originates from)
should not I strongly believe, be subverted to make any declaration about my entirely unrelated receiver policy.
(whether its my policy about receiving spam or bounces)
>It requires content filtering for airtightness, but is practical since <>
>is rare (and will stay that way if the filter is used), and it is
>reasonable to demand only one RCPT per transaction. A message can have
>many recipients but only one sender -- hence a bounce can only have one
>recipient.
we have enough unclean systems and systems incapable of even using spf v1 and dkim etc etc
backscatter and associated issues ONLY originate on these
subverting spf to say anything about reception/reply/bounce policy will not clean any of these or lessen the damage they do
(bandwith connections per minute etc. etc.)
but it will make all legitimate receivers, currently ONLY generating DSNs and NDRs to legitimate edge case senders (when mailbox full, and only to the first sender(as all later get the cached 5xx/4xx, first sender got the cached 2xx)
suddenly both non-compliant and require re-engineering working systems to become worse/broken to suppress NDRs and DSNs to the few senders(of the even fewer using any spf) that choose to publish such a policy
which helps none of the current victim receivers of backscatter, as 99% of backscatter is from broken systems (who dont/cant check spfv1) and to mainly domains not publishing any spf (as no intelligent spammer forges a from that can be detected via spf easilly)
spf is to cut down forgery by bad senders, to enable receivers to detect such forgery and reject
let the victims of backscatter report and deal with the bad senders, and sloppy receivers
(for most it is their only incentive to add SPF to stop their domains being easily forged)
if you wish to help fix the issue shun such bad receivers and senders by using backscatter dnsbls with appropriate responses (with exceptions when/if your users need them) and ensuring your systems arn't causing unnecessary NDRs and DSNs
but a new protocol that breaks good systems is not going to help anyone
broken systems cant be fixed by anyone but their owners, we can only give them incentive to fix their systems by refusing their users mail till their users force them to change.
this is how we fixed all the open relays,
this is how we'll fix senders that allow forged mail out
this is how we'll fix receivers that accept mail our SPF fail told them not to, then bounce the forged spam back to us.
if they didn't follow your spf policy to reject the mail, then why would they now stop sending the bounces.
if your spf ends ?all then you claim that your user might be sending from outside your ip's why should that user then not receive any legitimate bounces?
if your spf ends ~all then (well ive never claimed to understand what a receiver should do, for me its mark it for spamfolder even if it passes content filters.) still no reason to complain about later bounces as you didn't tell them to reject
but equally no excuse for not accepting the bounces