Discussion:
Solving the Forwarding Problem by Implementing SRS in Postfix
(too old to reply)
Meng Weng Wong
2004-01-11 19:17:28 UTC
Permalink
Note: I would prefer spf-discuss to focus on the challenge of adoption.
If you can help implement sender rewriting in an MTA, please join the
spf-devel list.

These are my thoughts on a strategic response to the forwarding problem.

Postfix-users has been going back and forth on the SPF issue. They have
identified forwarding as the biggest obstacle to SPF. I proposed sender
rewriting (http://spf.pobox.com/srs.html) and asked what it would take
to implement in Postfix. I personally believe that SRS is a horrible
hack, that it is terribly inelegant, and can lead to monstrously long
return paths if a message passes through multiple forwarding providers.
I also believe that the choice between 1.6 billion spams per day and an
ugly return-path is a no-brainer.

Schemes involving cryptography are the long-term solution. SPF may only
only be a step along the way, but it is a necessary step. I expect the
SPF syntax and architecture to outlive the "a mx ptr" mechanisms, as we
move toward next-generation mechanisms like "smime" and "gpg" and "envid".

In twenty years I expect to see signature credentials being passed in
the message headers or body or envelope. That will require upgrading
everyone's MTAs. And when those credentials are being passed,
forwarding will work again. Until then, I expect this to happen:

1) organizational forwarding providers like acm.org, pobox.com, and
alumniconnections will implement SRS. They have people on their
payrolls to do just that sort of thing.

2) major ISPs which provide forwarding services will implement SRS.

3) proactive / cutting edge / early-adopter / small hobbyist sites that
support forwarding will also implement SRS, by applying patches to
their MTA. We need to start working on SRS patches for MTAs so they
can do this.

4) MTA admins at sites that don't care about forwarding will watch
forwarding break. But this is okay! Why is this okay? Because the
ordinary end user population doen't care that an /etc/aliases entry
has stopped working. The ordinary end user population cares that
their mail gets through. I am counting on them to adapt: if a
forwarding address bounces, they will figure out an alternative email
address for the person, and they will use that instead. They won't
go on a campaign to convince everyone that classical forwarding as we
know it is an absolute moral good and must be defended at all costs.
(Well, some people will, but they tend to get ignored.) Forwarding
is just one way of doing things that people have gotten used to. It
will break for a little while, and then, when we move past designated
sender schemes, it will start working again.

So now what the world needs is SRS patches to MTAs. Here are some
thoughts from the postfix list:

On Sat, Jan 10, 2004 at 11:35:11AM -0500, Victor.Duchovni-/PgpppG8B+R7qynMiXIxWgC/***@public.gmane.org wrote:
| On Sat, 10 Jan 2004, Wietse Venema wrote:
|
| > That would be too late. The SMTP server must know whether an RCPT
| > TO or MAIL FROM presents a valid address before accepting the mail.
| >
|
| Also the counter in the cookie cannot reasonably scale to more than one MX
| host without introducing unreasonable requirements for centralized
| storage.
|
| 1. I am not likely to either implement SPF or say that SPF is a good idea
| any time soon.
|
| 2. This said, the question of *when* to perform SPF rewriting if such
| rewriting were to be done wisely or otherwise can be made clearer:
|
| - SRS Rewriting is only performed when the recipient address is
| changed in transit through the current host. So backup MX hosts
| do no rewriting, they are not forwarders, they are relays.
|
| - Also gateway systems do no rewriting, even when they do rewrite
| the envelope recipient, when they perform gatewaying for the
| destination system.
|
| - Virtual alias domains may have recipients in related domains
| behind a gateway (corporate deployment) or recipients in
| unaffiliated domains (ISP deployment). Postfix would have to
| distinguish between the two cases, presumably the "smtp"
| transport would SPF, and the "relay" transport would not.
|
| - SRS Rewriting is only performed when the original sender is
| intended to receive bounces, no such rewriting is required for
| mailing list submissions. Rewriting is required for alias
| expansions lacking a local "owner-" alias.
|
| - Hosts may not apply SPF restrictions when receiving mail from
| backup MX hosts or gateways associated with their organization.
| Only mail originating from suitably defined external systems
| that is again to be forwarded needs SRS sender rewriting.
|
| ...
|
| The implementation logic in the delivery agent would be something along
| the lines of:
|
| if (strcmp(rcpt, orcpt) != 0 /* Potential forwarding */
| && var_srs_rewriting /* Yes in "smtp" no in "relay" */
| && is_spf_domain(sender) /* Domain is not ours and does SPF */
| && !trusted_origin(origin) /* Did message orinate from "outside" */
| && .. more conditions? ...)
| {
| srs_sender = srs_rewrite(sender);
| }
|
| The trusted_origin() test needs to be performed accurately and is hard to
| get right. The SPF domain status lookup may fail, so one would have to
| defer the message until the sender domain status can be verified. Saving
| this in the queue file when the message is accepted can be wrong, because
| the message can be in the queue a long time, and the domain SPF records
| may change in the mean-time, so the status likely needs to be looked
| during delivery.
|
| Wietse's point about egress sender validation applies, since
|
| bounce+remote+cookie-***@public.gmane.org
|
| cannot be easily checked on egress. And of course on ingress the
| bounce+remote+cookie-***@public.gmane.org address would need to be checked in the SMTP
| daemon (another policy server).
|
| Growth in the sender address along the forwarding path can get quite
| unreasonable, but once all the sites along the path use SRS cookies the
| address cannot be trimmed without saving state and introducing a central
| data store for cooperating MX hosts. If there was a *standard* SRS
| encoding acceptable to all, one could save just the previous hop's cookie,
| this is not likely to happen any time soon and imposes unreasonable
| constraints on the structure of the *local* part of the address whose
| semantics have always been *local*.
|
| If I were to bet money on the likelihood of widespread interoperable
| robust SPF/SRS adoption within the next 5 years, I would bet against it
| even at unfavourable odds...
|
| --
| Viktor.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Greg Connor
2004-01-12 07:52:40 UTC
Permalink
--Meng Weng Wong <mengwong-GXA0fb4l1L89V/L/***@public.gmane.org> wrote:

> Note: I would prefer spf-discuss to focus on the challenge of adoption.
> If you can help implement sender rewriting in an MTA, please join the
> spf-devel list.


I'm replying here because these are not really "how to implement" but more
policy questions.

I have a semi-related question. Using SRS, is it possible to "fake" a
bounce, to cause the forwarder to think it's a bounce from something it
sent, when in reality the spammer is using the forwarding system to relay
to anyone and everyone? Or is that what the cookie is all about?

In terms of cookies, you could either a. keep track of outgoing cookies
used, or b. use a hash of the address-before-rewriting plus a "secret" so
that you can check the validity of the cookie without storing all cookies
given out so far. The hash approach is probably better for sites that have
multiple forwarding servers and don't want to store cookies in a database.


Second, a comment on SRS and forwarding policy:

Victor.Duchovni-/PgpppG8B+R7qynMiXIxWgC/***@public.gmane.org wrote:
| Growth in the sender address along the forwarding path can get quite
| unreasonable, but once all the sites along the path use SRS cookies the
| address cannot be trimmed without saving state and introducing a central
| data store for cooperating MX hosts. If there was a *standard* SRS
| encoding acceptable to all, one could save just the previous hop's cookie,
| this is not likely to happen any time soon and imposes unreasonable
| constraints on the structure of the *local* part of the address whose
| semantics have always been *local*.


I'm not sure if the return path really needs to grow arbitrarily long.
Really the *main* reason for rewriting the envelope sender is so that the
forwarded message can be acceptable to SPF checkers independent of the
sender's domain (since the sender doesn't know his mail is being forwarded,
and there are many senders).

Here is what I am thinking. Take an example of a message forwarded twice.
(copied from spf site)

Original message: from <joe-***@public.gmane.org> to <***@pobox.com>
Forwarded once: from <bounce-joe#aol.com-bob-e+***@public.gmane.org> to <***@mail.com>
Forward again: from <bounce-joe#aol.com-bob#pobox.com-srs-***@public.gmane.org> to
<oops-/***@public.gmane.org>
Or it could be: from <bounce-joe#aol.com-srs-***@public.gmane.org> to <***@yahoo.com>

This is part of the original SRS proposal and I think the idea is a good
one... any site that uses SRS should be able to recognize addresses that
have already been through SRS once and simplify the path -- that is, any
bounce can go directly back to the original sender, it doesn't need to
completely reverse the chain of how it was forwarded.

This is closer to how bounces work now, so we're not really losing anything
by doing this. Right now messages get forwarded multiple times without
rewriting and bounces just go directly back to the original sender.

However, on the other hand, from a pragmatic view it's probably easier to
implement SRS if you leave the entire original sender alone and not try to
analyze whether it was also an SRS address. For this method you would
replace the @ with #, and add -cookie-rnQS82vRMgMBgb4zQEcDBVaTQe2KTcn/@public.gmane.org" on the end.
Incoming messages from <> to bounce.forwarder.com would check cookie and
reverse the process. Probably shortening the path would take extra work on
the forwarding side.

So... how long can a return path be, and how many forwards do most people
expect to have? Is it worth the effort of trimming the previous SRS info
back?

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Kelson Vibber
2004-01-12 17:46:26 UTC
Permalink
At 11:52 PM 1/11/2004, Greg Connor wrote:
>Using SRS, is it possible to "fake" a bounce, to cause the forwarder to
>think it's a bounce from something it sent, when in reality the spammer is
>using the forwarding system to relay to anyone and everyone? Or is that
>what the cookie is all about?

Yes, that's the purpose of the cookie. If you receive something that looks
like a bounce but doesn't have the appropriate cookie, you assume it's been
faked and just discard it instead of passing it along.

I'll leave the rest for someone more familiar with SRS.

Kelson Vibber
SpeedGate Communications <www.speed.net>


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-17 15:14:37 UTC
Permalink
Why is this turning out to be such a big problem when there is such a
simple solution to the problem? It's called the "Sender" field in the
header and it's been there since day one, RFC 821.

Forget SRS, it's not needed.

Forwarders should always add a "Sender" header record because the they
are sending the forwarded message.

Mailing lists should use the "Sender" since they are sending the
message...

MSA's should always add a "Sender" header when the "From" field is not
the same as the authenticated sender.

SPF tests should always use the "Sender" if there present, not the
"From".

So the SPF check should work as follows.

If there is a "Sender" use it for checking and ignore the "From" else
use "From".

Problem solved, now I can use my AOL address when sending from another
ISP without having an issue with SPF. The "From" is not forged if the
"sender" is present.

So Meng, can you update the SPF docs to include this and then we can
put this one to bed and move on to other topics.



On 11 Jan 2004 at 14:17, Meng Weng Wong wrote:

> Note: I would prefer spf-discuss to focus on the challenge of adoption.
> If you can help implement sender rewriting in an MTA, please join the
> spf-devel list.
>
> These are my thoughts on a strategic response to the forwarding problem.
>
> Postfix-users has been going back and forth on the SPF issue. They have
> identified forwarding as the biggest obstacle to SPF. I proposed sender
> rewriting (http://spf.pobox.com/srs.html) and asked what it would take
> to implement in Postfix. I personally believe that SRS is a horrible
> hack, that it is terribly inelegant, and can lead to monstrously long
> return paths if a message passes through multiple forwarding providers.
> I also believe that the choice between 1.6 billion spams per day and an
> ugly return-path is a no-brainer.
>
> Schemes involving cryptography are the long-term solution. SPF may only
> only be a step along the way, but it is a necessary step. I expect the
> SPF syntax and architecture to outlive the "a mx ptr" mechanisms, as we
> move toward next-generation mechanisms like "smime" and "gpg" and "envid".
>
> In twenty years I expect to see signature credentials being passed in
> the message headers or body or envelope. That will require upgrading
> everyone's MTAs. And when those credentials are being passed,
> forwarding will work again. Until then, I expect this to happen:
>
> 1) organizational forwarding providers like acm.org, pobox.com, and
> alumniconnections will implement SRS. They have people on their
> payrolls to do just that sort of thing.
>
> 2) major ISPs which provide forwarding services will implement SRS.
>
> 3) proactive / cutting edge / early-adopter / small hobbyist sites that
> support forwarding will also implement SRS, by applying patches to
> their MTA. We need to start working on SRS patches for MTAs so they
> can do this.
>
> 4) MTA admins at sites that don't care about forwarding will watch
> forwarding break. But this is okay! Why is this okay? Because the
> ordinary end user population doen't care that an /etc/aliases entry
> has stopped working. The ordinary end user population cares that
> their mail gets through. I am counting on them to adapt: if a
> forwarding address bounces, they will figure out an alternative email
> address for the person, and they will use that instead. They won't
> go on a campaign to convince everyone that classical forwarding as we
> know it is an absolute moral good and must be defended at all costs.
> (Well, some people will, but they tend to get ignored.) Forwarding
> is just one way of doing things that people have gotten used to. It
> will break for a little while, and then, when we move past designated
> sender schemes, it will start working again.
>
> So now what the world needs is SRS patches to MTAs. Here are some
> thoughts from the postfix list:
>
> On Sat, Jan 10, 2004 at 11:35:11AM -0500, Victor.Duchovni-/PgpppG8B+R7qynMiXIxWgC/***@public.gmane.org wrote:
> | On Sat, 10 Jan 2004, Wietse Venema wrote:
> |
> | > That would be too late. The SMTP server must know whether an RCPT
> | > TO or MAIL FROM presents a valid address before accepting the mail.
> | >
> |
> | Also the counter in the cookie cannot reasonably scale to more than one MX
> | host without introducing unreasonable requirements for centralized
> | storage.
> |
> | 1. I am not likely to either implement SPF or say that SPF is a good idea
> | any time soon.
> |
> | 2. This said, the question of *when* to perform SPF rewriting if such
> | rewriting were to be done wisely or otherwise can be made clearer:
> |
> | - SRS Rewriting is only performed when the recipient address is
> | changed in transit through the current host. So backup MX hosts
> | do no rewriting, they are not forwarders, they are relays.
> |
> | - Also gateway systems do no rewriting, even when they do rewrite
> | the envelope recipient, when they perform gatewaying for the
> | destination system.
> |
> | - Virtual alias domains may have recipients in related domains
> | behind a gateway (corporate deployment) or recipients in
> | unaffiliated domains (ISP deployment). Postfix would have to
> | distinguish between the two cases, presumably the "smtp"
> | transport would SPF, and the "relay" transport would not.
> |
> | - SRS Rewriting is only performed when the original sender is
> | intended to receive bounces, no such rewriting is required for
> | mailing list submissions. Rewriting is required for alias
> | expansions lacking a local "owner-" alias.
> |
> | - Hosts may not apply SPF restrictions when receiving mail from
> | backup MX hosts or gateways associated with their organization.
> | Only mail originating from suitably defined external systems
> | that is again to be forwarded needs SRS sender rewriting.
> |
> | ...
> |
> | The implementation logic in the delivery agent would be something along
> | the lines of:
> |
> | if (strcmp(rcpt, orcpt) != 0 /* Potential forwarding */
> | && var_srs_rewriting /* Yes in "smtp" no in "relay" */
> | && is_spf_domain(sender) /* Domain is not ours and does SPF */
> | && !trusted_origin(origin) /* Did message orinate from "outside" */
> | && .. more conditions? ...)
> | {
> | srs_sender = srs_rewrite(sender);
> | }
> |
> | The trusted_origin() test needs to be performed accurately and is hard to
> | get right. The SPF domain status lookup may fail, so one would have to
> | defer the message until the sender domain status can be verified. Saving
> | this in the queue file when the message is accepted can be wrong, because
> | the message can be in the queue a long time, and the domain SPF records
> | may change in the mean-time, so the status likely needs to be looked
> | during delivery.
> |
> | Wietse's point about egress sender validation applies, since
> |
> | bounce+remote+cookie-***@public.gmane.org
> |
> | cannot be easily checked on egress. And of course on ingress the
> | bounce+remote+cookie-***@public.gmane.org address would need to be checked in the SMTP
> | daemon (another policy server).
> |
> | Growth in the sender address along the forwarding path can get quite
> | unreasonable, but once all the sites along the path use SRS cookies the
> | address cannot be trimmed without saving state and introducing a central
> | data store for cooperating MX hosts. If there was a *standard* SRS
> | encoding acceptable to all, one could save just the previous hop's cookie,
> | this is not likely to happen any time soon and imposes unreasonable
> | constraints on the structure of the *local* part of the address whose
> | semantics have always been *local*.
> |
> | If I were to bet money on the likelihood of widespread interoperable
> | robust SPF/SRS adoption within the next 5 years, I would bet against it
> | even at unfavourable odds...
> |
> | --
> | Viktor.
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren, President | Tel.: +1 714-573-9650
Warren Engineering | Fax: +1 714-573-9289
14681 Danborough Rd. | mailto:jwarren-***@public.gmane.org
Tustin, CA 92780-6755 | mailto:info-***@public.gmane.org
| http:Someday.com
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Alain Knaff
2004-01-17 15:24:28 UTC
Permalink
begin Saturday 17 January 2004 16:14, John Warren quote:
> SPF tests should always use the "Sender" if there present, not the
> "From".
>
> So the SPF check should work as follows.
>
> If there is a "Sender" use it for checking and ignore the "From" else
> use "From".

The problem with this approach (and others approaches based on header
fields, rather than the envelope) is that it forces the MTA to
actually accept the message, before being able to reject it.

With SPF as it stands now, the mail transaction can already be aborted
before a single byte of message data is transferred.

Alain


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-17 15:37:26 UTC
Permalink
On 17 Jan 2004 at 16:24, Alain Knaff wrote:

> begin Saturday 17 January 2004 16:14, John Warren quote:
> > SPF tests should always use the "Sender" if there present, not the
> > "From".
> >
> > So the SPF check should work as follows.
> >
> > If there is a "Sender" use it for checking and ignore the "From" else
> > use "From".
>
> The problem with this approach (and others approaches based on header
> fields, rather than the envelope) is that it forces the MTA to
> actually accept the message, before being able to reject it.
>
> With SPF as it stands now, the mail transaction can already be aborted
> before a single byte of message data is transferred.
>
> Alain
>

Not true, but I did fail to talk about the transaction header as well.
Thanks for bringing it up.

The "RCPT TO:" in the transaction header should always the
authenticated sender e-mail address and never the address the user
supplied in the "From" since it could be forged.

The "Sender" field, which should match the "RCPT TO:", would be used
for later testing if you don't test at transaction time.

So the problem is still solved.

Next comment please:)



> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Dan Boresjo
2004-01-17 18:16:32 UTC
Permalink
On Saturday 17 January 2004 3:37 pm, John Warren wrote:
> The "RCPT TO:" [sic] in the transaction header should always the
> authenticated sender e-mail address and never the address the user
> supplied in the "From" since it could be forged.

It isn't. My ISP (using exim) lets me use any 'From:' address I want and sets
the MAIL FROM accordingly. They don't enforce any 'Sender:' field at all.

Interestingly, they also send bounces to the third-party address too, which
implies they completely ignore the original source of the message...

- Dan

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-17 15:40:24 UTC
Permalink
WHAT WAS I THINK ON MY LAST MESSAGE?

I meant the "MAIL FROM:" transaction field not the "RCPT TO:" field.

Sorry about that.


On 17 Jan 2004 at 16:24, Alain Knaff wrote:

> begin Saturday 17 January 2004 16:14, John Warren quote:
> > SPF tests should always use the "Sender" if there present, not the
> > "From".
> >
> > So the SPF check should work as follows.
> >
> > If there is a "Sender" use it for checking and ignore the "From" else
> > use "From".
>
> The problem with this approach (and others approaches based on header
> fields, rather than the envelope) is that it forces the MTA to
> actually accept the message, before being able to reject it.
>
> With SPF as it stands now, the mail transaction can already be aborted
> before a single byte of message data is transferred.
>
> Alain
>
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Mark
2004-01-17 16:17:17 UTC
Permalink
----- Original Message -----
From: "John Warren" <John-***@public.gmane.org>
To: <spf-devel-***@public.gmane.org>; <spf-***@v2.listbox.com>
Sent: Saturday, January 17, 2004 4:15 PM
Subject: [spf-discuss] Solving the Forwarding Problem for good!!!

> Why is this turning out to be such a big problem when there is such a
> simple solution to the problem? It's called the "Sender" field in the
> header and it's been there since day one, RFC 821.
>
> Forget SRS, it's not needed.
>
> Forwarders should always add a "Sender" header record because the they are
> sending the forwarded message.
>
> Mailing lists should use the "Sender" since they are sending the
> message...
>
> MSA's should always add a "Sender" header when the "From" field is not the
> same as the authenticated sender.
>
> SPF tests should always use the "Sender" if there present, not the "From".
>
> So the SPF check should work as follows.
>
> If there is a "Sender" use it for checking and ignore the "From" else use
> "From".
>
> Problem solved, now I can use my AOL address when sending from another
> ISP without having an issue with SPF. The "From" is not forged if the
> "sender" is present.
>
> So Meng, can you update the SPF docs to include this and then we can
> put this one to bed and move on to other topics.

As Alain already pointed out to you:

> > The problem with this approach (and others approaches based on header
> > fields, rather than the envelope) is that it forces the MTA to
> > actually accept the message, before being able to reject it.
> >
> > With SPF as it stands now, the mail transaction can already be aborted
> > before a single byte of message data is transferred.

Yes. After the DATA phase, the message can still be rejected (like an
anti-virus milter would do, for instance), but not DURING the data phase.
So, once you negotiated the start of the DATA phase, you will then have to
slurp in the entire message first, which is rather inefficient for SPF
checks.

Other approaches based on header fields, rather than the envelope, also
suffer from possible replay-attacks; so, a signing scheme of sorts would
have to accompany them. I suggest you browse the archives a bit, John;
because much has already been written on this.

Your post certainly radiates enthusiasm, John. :) But, the odd Eureka moment
notwithstanding, I think, in general, it is fair to subject ourselves to the
sanity check of realizing that if hundreds of people have been looking for a
solution for hundreds of hours, that any brilliant solution that pops up in
our head within seconds, has probably already long since been considered,
and discarded, or frowned upon, for reasons we may not have immediately
thought of.

P.S. I took "spf-devel" off the cc; it has no place there.

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-17 16:49:00 UTC
Permalink
I don't think you read my second message, which I made and error in as
stated in a follow up.

I think the "MAIL FROM:" transaction field should contain the
authenticated sender address not the field supplied by the user in the
"From" header field. The "MAIL FROM:" would then be the same as the
"Sender" header field.

Who is the true sender of the message? It has to the the authenticated
sender not the "From" sender which could be forged even if it is a
legal forgery.

I don't remember every seeing the contents of the "MAIL FROM:"
transaction header every being passed on in the delivered message in
any field that a mail client would display. So it makes since that the
"MAIL FROM:" should the the authenticated sender true e-mail address.

This would solve the issue and not be a kludge like SRS plus it uses
all standard fields.

Oh one more point, you can abort during the "DATA" phase, you don't
have to accept the entire message. But in this case that would not be
required.



On 17 Jan 2004 at 16:17, Mark wrote:

> ----- Original Message -----
> From: "John Warren" <John-***@public.gmane.org>
> To: <spf-devel-***@public.gmane.org>; <spf-***@v2.listbox.com>
> Sent: Saturday, January 17, 2004 4:15 PM
> Subject: [spf-discuss] Solving the Forwarding Problem for good!!!
>
> > Why is this turning out to be such a big problem when there is such a
> > simple solution to the problem? It's called the "Sender" field in the
> > header and it's been there since day one, RFC 821.
> >
> > Forget SRS, it's not needed.
> >
> > Forwarders should always add a "Sender" header record because the they are
> > sending the forwarded message.
> >
> > Mailing lists should use the "Sender" since they are sending the
> > message...
> >
> > MSA's should always add a "Sender" header when the "From" field is not the
> > same as the authenticated sender.
> >
> > SPF tests should always use the "Sender" if there present, not the "From".
> >
> > So the SPF check should work as follows.
> >
> > If there is a "Sender" use it for checking and ignore the "From" else use
> > "From".
> >
> > Problem solved, now I can use my AOL address when sending from another
> > ISP without having an issue with SPF. The "From" is not forged if the
> > "sender" is present.
> >
> > So Meng, can you update the SPF docs to include this and then we can
> > put this one to bed and move on to other topics.
>
> As Alain already pointed out to you:
>
> > > The problem with this approach (and others approaches based on header
> > > fields, rather than the envelope) is that it forces the MTA to
> > > actually accept the message, before being able to reject it.
> > >
> > > With SPF as it stands now, the mail transaction can already be aborted
> > > before a single byte of message data is transferred.
>
> Yes. After the DATA phase, the message can still be rejected (like an
> anti-virus milter would do, for instance), but not DURING the data phase.
> So, once you negotiated the start of the DATA phase, you will then have to
> slurp in the entire message first, which is rather inefficient for SPF
> checks.
>
> Other approaches based on header fields, rather than the envelope, also
> suffer from possible replay-attacks; so, a signing scheme of sorts would
> have to accompany them. I suggest you browse the archives a bit, John;
> because much has already been written on this.
>
> Your post certainly radiates enthusiasm, John. :) But, the odd Eureka moment
> notwithstanding, I think, in general, it is fair to subject ourselves to the
> sanity check of realizing that if hundreds of people have been looking for a
> solution for hundreds of hours, that any brilliant solution that pops up in
> our head within seconds, has probably already long since been considered,
> and discarded, or frowned upon, for reasons we may not have immediately
> thought of.
>
> P.S. I took "spf-devel" off the cc; it has no place there.
>
> - Mark
>
> System Administrator Asarian-host.org
>
> ---
> "If you were supposed to understand it,
> we wouldn't call it code." - FedEx
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
wayne
2004-01-17 17:19:07 UTC
Permalink
In <4008F6F4.30978.9F7CA2A8-bi+***@public.gmane.org> "John Warren" <***@wenet.tustin.ca.us> writes:

> I think the "MAIL FROM:" transaction field should contain the
> authenticated sender address not the field supplied by the user in the
> "From" header field. The "MAIL FROM:" would then be the same as the
> "Sender" header field.

It is my understanding that the Sender: header is, unfortunately,
ambiguous. It's exact defininition changed between RFC822 and RFC2822
and RFC822 used it in two different ways. I forget where I saw this
discussed, but I remember being pretty well convinced that Sender: is
not very reliable.

> Who is the true sender of the message? It has to the the authenticated
> sender not the "From" sender which could be forged even if it is a
> legal forgery.

A more likely candidate for what you describe would be Resent-From:
however I think this is not always reliable either.


> I don't remember every seeing the contents of the "MAIL FROM:"
> transaction header every being passed on in the delivered message in
> any field that a mail client would display. So it makes since that the
> "MAIL FROM:" should the the authenticated sender true e-mail address.

The envelope-from is often contained in such headers as Envelope-From:
and Return-Path:.


> This would solve the issue and not be a kludge like SRS plus it uses
> all standard fields.

None of the standard fields have a cookie that will allow a system to
reliable tell if it should accept a bounce or not.


> Oh one more point, you can abort during the "DATA" phase, you don't
> have to accept the entire message. But in this case that would not be
> required.

I can't see how you can abort during the DATA phase, there is no
communication from the receiver until the final ".<NL><CR>" is sent.

It would have been really nice if SMTP had a HEADERS and BODY pair of
commands instead of a DATA command, but it is a little late to change
that now.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Mark
2004-01-17 17:56:52 UTC
Permalink
----- Original Message -----
From: "John Warren" <John-***@public.gmane.org>
To: <spf-discuss-***@public.gmane.org>
Sent: Saturday, January 17, 2004 5:49 PM
Subject: Re: [spf-discuss] Solving the Forwarding Problem for good!!!

> I think the "MAIL FROM:" transaction field should contain the
> authenticated sender address not the field supplied by the user in the
> "From" header field.

Headers are part of the DATA stream. They have, as far as the SMTP protocol
is concerned, nothing to do with the envelope sender. The envelope from is
often, on mailing-lists, for instance, radically different from whatever
"From:" address is to be found in the headers. Mind you, this does not
actually impact SPF negatively in any way, as SPF deals with the domain of
the envelope from address, and not its local part.

> I don't remember every seeing the contents of the "MAIL FROM:"
> transaction header every being passed on in the delivered message in
> any field that a mail client would display.

Then you have not been looking properly. :) In your sendmail.cf it would be
the $g macro, which you might find in the Return-Path:, or as part of a
Received: header,

H?P?Return-Path: <$g>
HReceived: $?sfrom $s $.$?_($?s$|from $.$_)
$.$?{auth_type}(authenticated$?{auth_ssf} bits=${auth_ssf}$.)
...
$.$b$?g
(envelope-from $g)$.

In real life, for instance,

Received: from sc8-sf-list1.sourceforge.net (lists.sourceforge.net
[66.35.250.206])
by asarian-host.net (8.12.10/8.12.10) with ESMTP id i0HHJQfR004779
(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO);
Sat, 17 Jan 2004 18:19:27 +0100 (CET)
(envelope-from spamassassin-talk-admin-5NWGOfrQmneRv+***@public.gmane.org)

None of that is relevant, really; as a Milter gets the envelope from nicely
parsed to itself, as has no need to extract it from any header field.

> Oh one more point, you can abort during the "DATA" phase, you don't have
> to accept the entire message.

Nope; you can drop the connection, of course, and have the connecting mail
server time out; but then you're probably even worse off, as the connecting
mail server will just keep trying to deliver the message. Other than that,
you will just have to wait the process out, until /^\.$/ (in Perl terms).

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Dan Boresjo
2004-01-17 18:09:38 UTC
Permalink
On Saturday 17 January 2004 5:56 pm, Mark wrote:
> Nope; you can drop the connection, of course, and have the connecting mail
> server time out; but then you're probably even worse off, as the connecting
> mail server will just keep trying to deliver the message. Other than that,
> you will just have to wait the process out, until /^\.$/ (in Perl terms).

Dropping the connection _and_ a one-time blacklisting of the sending IP works,
as the sender gives up after getting a "550 Blacklisted" the next time it
connects.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Mark
2004-01-17 18:28:38 UTC
Permalink
----- Original Message -----
From: "Dan Boresjo" <dan-8EeTylryM6IG2Il/***@public.gmane.org>
To: <spf-discuss-***@public.gmane.org>
Sent: Saturday, January 17, 2004 7:08 PM
Subject: Re: [spf-discuss] Solving the Forwarding Problem for good!!!

> On Saturday 17 January 2004 5:56 pm, Mark wrote:
>
> > Nope; you can drop the connection, of course, and have the connecting
> > mail server time out; but then you're probably even worse off, as the
> > connecting mail server will just keep trying to deliver the message.
> > Other than that, you will just have to wait the process out, until
> > /^\.$/ (in Perl terms).
>
> Dropping the connection _and_ a one-time blacklisting of the sending IP
> works, as the sender gives up after getting a "550 Blacklisted" the next
> time it connects.

Yes, that might work (although the sender could be trying to send a new
message, the next time; not all that uncommon with spam). But such a scheme,
like grey-listing, stretches a bit beyond the standard SMTP protocol, of
course. :)

As wayne said, it would have been great if a separate HEADERS and BODY
command existed. The NNTP protocol, for example, has all three: ARTICLE,
BODY, and HEAD.

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Phil Howard
2004-01-17 21:36:46 UTC
Permalink
On Sat, Jan 17, 2004 at 08:49:00AM -0800, John Warren wrote:

| I think the "MAIL FROM:" transaction field should contain the
| authenticated sender address not the field supplied by the user in the
| "From" header field. The "MAIL FROM:" would then be the same as the
| "Sender" header field.
|
| Who is the true sender of the message? It has to the the authenticated
| sender not the "From" sender which could be forged even if it is a
| legal forgery.

And "authenticated sender" means what? Would that be the address of the
local mailbox from which automated forwarding comes from?


| I don't remember every seeing the contents of the "MAIL FROM:"
| transaction header every being passed on in the delivered message in
| any field that a mail client would display. So it makes since that the
| "MAIL FROM:" should the the authenticated sender true e-mail address.
|
| This would solve the issue and not be a kludge like SRS plus it uses
| all standard fields.

I'll try to re-iterate one of the several ideas that popped into my head
based on what you said. I'll pick the one that makes the most sense and
seems the most plausible as what you meant and what might work.

When a forwarder gets email addressed to an automatic forwarding address,
it will take the envelope sender (e.g. MAIL FROM) address on the incoming
transaction, and append an appropriate RFC822 header. Do you mean that
it should be "Sender:"? Then it would replace the envelope sender with
one that is true and authenticated. But the question is, what? The
original envelope sender, and any from the RFC822 headers, could be
forged. But is that OK anyway?

What about bounces that get sent to the envelope sender? Or is that not
allowed if an RFC822 Sender: header is present?

If a Sender already exists, should it be retained? replaced? added to?
If replaced, should it be changed to X-Sender: and retain that to effect
an addition without duplication?

If a mailing list follows this logic, what are the implications?


| Oh one more point, you can abort during the "DATA" phase, you don't
| have to accept the entire message. But in this case that would not be
| required.

It is preferred to not engage the DATA phase if at all possible. Had
your proposal really meant that it was required, it surely would not
be accepted.

--
-----------------------------------------------------------------------------
| Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ |
| (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ |
-----------------------------------------------------------------------------

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-18 03:20:14 UTC
Permalink
On 17 Jan 2004 at 15:36, Phil Howard wrote:

> On Sat, Jan 17, 2004 at 08:49:00AM -0800, John Warren wrote:
>
> | I think the "MAIL FROM:" transaction field should contain the
> | authenticated sender address not the field supplied by the user in the
> | "From" header field. The "MAIL FROM:" would then be the same as the
> | "Sender" header field.
> |
> | Who is the true sender of the message? It has to the the authenticated
> | sender not the "From" sender which could be forged even if it is a
> | legal forgery.
>
> And "authenticated sender" means what? Would that be the address of the
> local mailbox from which automated forwarding comes from?

In the case of user mail sent directly that would be their local
mailbox address, with domain.

For forwarded mail it would be the local mailbox address. After all
that's the sender even though it's automated.

For a mailing list it's the mailing list address.

>
>
> | I don't remember every seeing the contents of the "MAIL FROM:"
> | transaction header every being passed on in the delivered message in
> | any field that a mail client would display. So it makes since that the
> | "MAIL FROM:" should the the authenticated sender true e-mail address.
> |
> | This would solve the issue and not be a kludge like SRS plus it uses
> | all standard fields.
>
> I'll try to re-iterate one of the several ideas that popped into my head
> based on what you said. I'll pick the one that makes the most sense and
> seems the most plausible as what you meant and what might work.
>
> When a forwarder gets email addressed to an automatic forwarding address,
> it will take the envelope sender (e.g. MAIL FROM) address on the incoming
> transaction, and append an appropriate RFC822 header. Do you mean that
> it should be "Sender:"?

Yes, it should be the "Sender:" since the forwarder is the sender. The
"From:" remains unchanged.

> Then it would replace the envelope sender with
> one that is true and authenticated. But the question is, what? The
> original envelope sender, and any from the RFC822 headers, could be
> forged. But is that OK anyway?

Those addresses are passed untouched. We only deal with the "Sender:"
because we want to know who is the current sender on this mail server.
That should be an authenticated sender on the mail system sending the
message.

>
> What about bounces that get sent to the envelope sender? Or is that not
> allowed if an RFC822 Sender: header is present?

The bounce at the transaction phase would to the the "MAIL FROM:" since
that is where the message is coming from. Bounces later on would go as
required by the RFCs.

>
> If a Sender already exists, should it be retained? replaced? added to?
> If replaced, should it be changed to X-Sender: and retain that to effect
> an addition without duplication?

That's a good question and I don't have a good answer but only my
opinion.

I think there should only be one "Sender:" in the header so either you
have to drop the older ones or change them in some way. Changing it to
"X-Sender:" and leaving them in their correct location compared with
the "Receive" headers is one way.

I think that this has already been addressed in RFC 2476.

----------------------------- RFC 2476 ------------------------------

8.1 Add 'Sender'
The MSA MAY add or replace the 'Sender' field, if the identity of the
sender is known and this is not given in the 'From' field.

The MSA MUST ensure that any address it places in a 'Sender' field is
in fact a valid mail address.

---------------------------------------------------------------------

So it's completely acceptable to replace the "Sender" field. I have
seen comments about this stating that renaming the previous sender to
"Old-Sender" could be a way to go but there is nothing said in the RFCs
on the issue. You could end up with several "Old-Sender"s.





>
> If a mailing list follows this logic, what are the implications?

Glad you brought that up. Take a look at this mail list. The "Sender:
is the mail list address and the "From:" is the address of the original
sender. Now what does the "MAIL FROM: look like. Well here is the
transaction log from messages from this mailing list.

T 20040117 071528 3ff3d6ae Connection from 208.58.1.195
T 20040117 071528 3ff3d6ae EHLO portent.listbox.com
T 20040117 071529 3ff3d6ae MAIL FROM:<owner-spf-discuss-***@public.gmane.org>
SIZE=11374
T 20040117 071530 3ff3d6ae RCPT TO:<john-***@public.gmane.org>
T 20040117 071531 3ff3d6ae DATA - 233 lines, 11372 bytes.
T 20040117 071531 3ff3d6ae QUIT
T 20040117 071531 3ff3d6ae Connection closed with 208.58.1.195, 3 sec.
elapsed.

As you can see the "MAIL FROM:" is the list owner which is as it should
be since that's where the message is being sent from.



>
>
> | Oh one more point, you can abort during the "DATA" phase, you don't
> | have to accept the entire message. But in this case that would not be
> | required.
>
> It is preferred to not engage the DATA phase if at all possible. Had
> your proposal really meant that it was required, it surely would not
> be accepted.
>

I agree that it's not the best place to abort but there are time when
the DATA phase does get aborted. Sending too much data is one place
I've seen mail system abort the DATA phase. In this case I see no
reason that it need be done.



The bottom line is that if you agree that the "MAIL FROM:" should
always be a valid sender on the mail system that the message is being
sent from, as it must from the reading of RFC 2476, then we need do
nothing to SPF other than point out what is valid to check and reject
on.

In this case what is being stated is that the "MAIL FROM:" e-mail
address should be an authenticated sender. If not and you have a "-all"
mail could be rejected.

Given that the next step would be to push the vendors to update their
mail system so that they always supply the senders valid authenticated
address in the "MAIL FROM:" and "Sender" records.

Now I can send my mail with my return address from any ISPs mail system
that I have authenticated access to which is as it should be.



> --
> -----------------------------------------------------------------------------
> | Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ |
> | (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ |
> -----------------------------------------------------------------------------
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Brian Hatch
2004-01-18 05:26:35 UTC
Permalink
> Yes. After the DATA phase, the message can still be rejected (like an
> anti-virus milter would do, for instance), but not DURING the data phase.
> So, once you negotiated the start of the DATA phase, you will then have to
> slurp in the entire message first, which is rather inefficient for SPF
> checks.

Hmmmn. A thought, and admittedly it's likely a bad one.

If for some valid reason you were not able to determine
if a message should be blocked before data and needed to
examine the header a bit, and once you read enough of the
content was able to determine that it should in fact be
dropped, you could

* close the connection immediately
* save the ehlo/mail from/rcpt to/client ip
as a 'ban next time' tuple
* when the remote server attempts to resend it,
block the email before DATA.

This would have a chance of blocking legit email, so it's
almost a guarenteed bad idea. But it could be useful in
the cases where you're fairly sure the tuple is unique and
won't have false positives. No need to slurp all the data
before rejecting. Has a 'greylisting' feel to it.


--
Brian Hatch No windows were
Systems and crashed in the
Security Engineer creation of this
http://www.ifokr.org/bri/ email message.

Every message PGP signed

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Dan Boresjo
2004-01-18 12:30:10 UTC
Permalink
On Sunday 18 January 2004 5:26 am, Brian Hatch wrote:
> Hmmmn. A thought, and admittedly it's likely a bad one.
>
> If for some valid reason you were not able to determine
> if a message should be blocked before data and needed to
> examine the header a bit, and once you read enough of the
> content was able to determine that it should in fact be
> dropped, you could
> * close the connection immediately
> * save the ehlo/mail from/rcpt to/client ip
> as a 'ban next time' tuple
> * when the remote server attempts to resend it,
> block the email before DATA.

This is what I have been doing for quite a while, it works. Then again I am
not really worried if the occasional legit mail gets bounced, because they
get a reject saying:

550 sending host is a known virus source

In my view their virus infection is something they need to know about and fix.

- Dan

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-18 16:04:03 UTC
Permalink
Let's stop and take a deep breath.

Ok, now that we done that I'm going to restate my idea since my
original one was on the right track but had some flaws and my adjusted
one seems not to have gotten across.

First I'd like to stick to the problem that needs to be solved and not
what the mail systems are doing now.

The idea of using SRS is going to require changes to MTAs which would
also be the case with what I'm purposing but I think my changes follow
the RFCs better than SRS.

I also think that my idea is already based on what the intent of RFCs
2476 and 2821 but is not currently being used as I think it should be.
I could be wrong.

----

Problem.

Being able to prove that mail is being sent from a valid mail host and
or user.

Solutions:

SPF and IMX plus others. IMX does take care of where mail comes from
and SPF does protect the domain owner. Both can and should work
together.

----

Problem:

Being able to send mail using your legal e-mail address from a domain
other than the one in your e-mail address. This has very basic and
valid business reasons and needs to not be blocked.

Users in the the field may need to send mail from their company when
they can not get that mail back to their company mail server. The only
way that they can send it is to use the ISP mail server they are
connecting to.

The problem is how to do this and still track the sending user. SPFs
filtering could cause valid mail of this type to be blocked.

Solution:

The Sending MTA/MSA must make sure that the "MAIL FROM" in the SMTP
envelope MUST always contain the valid local authenticated e-mail
address of the of the sender no matter what the sender puts in the
"FROM" or "REPLY-TO" fields in the message header. Both RFC 2476 and
2821should be updated to require this since it's currently ambiguous.

The Sending MTA Should replace or add, as required, a "SENDER" field to
the message header, as stated in RFC 2476, that matched the "MAIL
FROM" used in the SMTP envelope.

I think that RFC 2476 should be changed to make this a MUST rather than
SHOULD requirement.

----

It is understood that a mail server that runs as an open relay need not
do any of this but we all know what happens to open relay mail servers
so it's a non-issue.


It is my belief that this then solves a lot of issues. You have domain
protection in messages sent as SPF is trying to do because you know who
the real sender is.

You have the ability to send a message using your valid and legal
domain name from another domain. This can be done since since you now
know who sent the message even though they have legally forged their
"From" e-mail address.

Forwarding can always be done since you know who forwarded the message.

Mailing list can forward without problems since she sender, the list,
is know.

You can block all mail sent with a "MAIL FROM" that contains a domain
that the sending mail server is not listed under without any issues of
blocking good mail.

Win, Win, Win so what am I missing?











----------------------------------------------------------------------
John Warren, President | Tel.: +1 714-573-9650
Warren Engineering | Fax: +1 714-573-9289
14681 Danborough Rd. | mailto:jwarren-***@public.gmane.org
Tustin, CA 92780-6755 | mailto:info-***@public.gmane.org
| http:Someday.com
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
wayne
2004-01-18 17:36:26 UTC
Permalink
In <400A3DF2.24050.A479CC12-bi+***@public.gmane.org> "John Warren" <***@wenet.tustin.ca.us> writes:

> Let's stop and take a deep breath.
>
> Ok, now that we done that I'm going to restate my idea since my
> original one was on the right track but had some flaws and my adjusted
> one seems not to have gotten across.

Good idea.


> The idea of using SRS is going to require changes to MTAs which would
> also be the case with what I'm purposing but I think my changes follow
> the RFCs better than SRS.

That is a valid idea to consider.


> Problem.
>
> Being able to prove that mail is being sent from a valid mail host and
> or user.
>
> Solutions:
>
> SPF and IMX plus others. IMX does take care of where mail comes from
> and SPF does protect the domain owner. Both can and should work
> together.

Note: SPF does everything that IMX does. While they may be able to
work together, using both is redundant.

SPF, IMX, RMX, DMP, DRIP, etc. are all designated sender systems.
They don't say whether an IP address is a valid mail host. Rathery,
they say whether a given domain owner has authorized a given IP
address as being authorized to send email using their domain name.
All of these designated sender systems allow you to say that 224.1.2.3
is a valid IP address, but since 224.1.2.3 is a multicast IP address,
it can't possibly be a valid mail host.

MTAMark, on the otherhand, can be used to mark valid MTA hosts.


> Problem:
>
> Being able to send mail using your legal e-mail address from a domain
> other than the one in your e-mail address. This has very basic and
> valid business reasons and needs to not be blocked.
> [...]
>
> Solution:
>
> The Sending MTA/MSA must make sure that the "MAIL FROM" in the SMTP
> envelope MUST always contain the valid local authenticated e-mail
> address of the of the sender no matter what the sender puts in the
> "FROM" or "REPLY-TO" fields in the message header. Both RFC 2476 and
> 2821should be updated to require this since it's currently ambiguous.
>
> The Sending MTA Should replace or add, as required, a "SENDER" field to
> the message header, as stated in RFC 2476, that matched the "MAIL
> FROM" used in the SMTP envelope.
>
> I think that RFC 2476 should be changed to make this a MUST rather than
> SHOULD requirement.

Getting an RFC through IETF that would change RFC2476 and RFC2821 is
going to be *real* hard. Getting actual MTAs in the field to comply
with those changes will take decades.

Note that RFC2821 is still a *proposed standard*, it still needs to
get through the *draft standard* stage before it can supersede RFC821
as an *internet standard*. RFC2821 was written in Apr 2001, close to
three years ago. It will be lucky to make it through the final two
stages this year.


> Win, Win, Win so what am I missing?

I want something done within my lifetime. Your proposed changes to
RFCs will not be reflected in the real world for a *very* long time.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Alex van den Bogaerdt
2004-01-18 22:10:25 UTC
Permalink
On Sun, Jan 18, 2004 at 08:04:03AM -0800, John Warren wrote:

> Solution:
>
> The Sending MTA/MSA must make sure that the "MAIL FROM" in the SMTP
> envelope MUST always contain the valid local authenticated e-mail
> address of the of the sender no matter what the sender puts in the
> "FROM" or "REPLY-TO" fields in the message header. Both RFC 2476 and
> 2821should be updated to require this since it's currently ambiguous.
>
> The Sending MTA Should replace or add, as required, a "SENDER" field to
> the message header, as stated in RFC 2476, that matched the "MAIL
> FROM" used in the SMTP envelope.

Just to make sure I understand everything you say, and in the mean
time asking some questions. Is the following a correct interpretation
of your message, and can you answer the questions?

User <xyz-***@public.gmane.org> is on the road. He want to send a message
to <abc-iX3ng/***@public.gmane.org>, and do so from ISP "isp.tld" as user <***@isp.tld>

From pqr-***@public.gmane.org
Received-by ....
...etc...
From: "user" <xyz-***@public.gmane.org>
To: "other" <abc-iX3ng/***@public.gmane.org>
Sender: "user" <pqr-***@public.gmane.org>
...etc...

Bounces (if any) will go to pqr-***@public.gmane.org and replies will go to
xyz-***@public.gmane.org

So far so good. Now, this user wants to be able to read his mail.
At domain.tld, there is a forward setup. "Other" is replying to
the message and will do so to <xyz-***@public.gmane.org>. domain.tld is
accepting the message, the .forward is processed and mail is being
sent out again to <pqr-***@public.gmane.org>.

This will look like:

From xyz-***@public.gmane.org
Received-by ....
...etc...
From: "Other" <other-iX3ng/***@public.gmane.org>
To: "user" <xyz-***@public.gmane.org>
Sender: "user" <xyz-***@public.gmane.org>
...etc...

Bounces, if any, will go to <xyz-***@public.gmane.org> and not to <***@other.tld>
According to some, this is unacceptable. Consider this:

mail from: <xyz-***@public.gmane.org>
250 ...
rcpt to: <pqr-***@public.gmane.org>
5xx mailbox full

At this moment, I think the MTA at "domain.tld" could process the
bounce to <other-iX3ng/***@public.gmane.org> but this does currently not happen. It
will go to <xyz-***@public.gmane.org> at the moment.

Another possibility is that the bounce is not generated by "domain.tld"
but by "isp.tld" and the resulting mail will be introduced like this:

mail from: <>
rcpt to: <xyz-***@public.gmane.org>

Again, the bounce will be delivered to "xyz", not "other". Even worse,
many users will forward this bounce again, in stead of storing it
locally (good enough) or, even better, letting <other-iX3ng/***@public.gmane.org> know.

IMHO "domain.tld" accepted responsability for the message and should
therefore accept responibility for informing <other-iX3ng/***@public.gmane.org> about
any delivery problems.

It may be possible no direct communication can or may occur between
"isp.tld" and "other.tld" (for instance: blacklisting) so sending
the bounce from "isp.tld" to "other.tld" (as currently would happen
without SPF) is bad in it self.

Last but not least, "other.tld" does not expect a bounce from "isp.tld"
thus the bounce may end up inbetween all other spam. Would the bounce
have been sent by "domain.tld", it has a bigger chance to survive.

How does your proposal handle bounces? Can you provide an example
which includes forwarding, via a blind gateway that accepts messages
without being able to check if it can be delivered?

cheers,
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-19 04:24:44 UTC
Permalink
On 18 Jan 2004 at 11:36, wayne wrote:

> In <400A3DF2.24050.A479CC12-bi+***@public.gmane.org> "John Warren" <***@wenet.tustin.ca.us> writes:
>
> > Let's stop and take a deep breath.
> >
> > Ok, now that we done that I'm going to restate my idea since my
> > original one was on the right track but had some flaws and my adjusted
> > one seems not to have gotten across.
>
> Good idea.
>
>
> > The idea of using SRS is going to require changes to MTAs which would
> > also be the case with what I'm purposing but I think my changes follow
> > the RFCs better than SRS.
>
> That is a valid idea to consider.
>
>
> > Problem.
> >
> > Being able to prove that mail is being sent from a valid mail host and
> > or user.
> >
> > Solutions:
> >
> > SPF and IMX plus others. IMX does take care of where mail comes from
> > and SPF does protect the domain owner. Both can and should work
> > together.
>
> Note: SPF does everything that IMX does. While they may be able to
> work together, using both is redundant.

Not the way I thought I understood it. If your inbound mail server
also does all of your outbound mail you don't need a IMX but if you run
a split system then you need a IMX for you outbound servers.

It does some of the things that SPF does but not all.

It's kinda like a square is a rectangle but a rectangle is not a
square.

>
> SPF, IMX, RMX, DMP, DRIP, etc. are all designated sender systems.
> They don't say whether an IP address is a valid mail host. Rathery,
> they say whether a given domain owner has authorized a given IP
> address as being authorized to send email using their domain name.
> All of these designated sender systems allow you to say that 224.1.2.3
> is a valid IP address, but since 224.1.2.3 is a multicast IP address,
> it can't possibly be a valid mail host.
>
> MTAMark, on the otherhand, can be used to mark valid MTA hosts.
>
>
> > Problem:
> >
> > Being able to send mail using your legal e-mail address from a domain
> > other than the one in your e-mail address. This has very basic and
> > valid business reasons and needs to not be blocked.
> > [...]
> >
> > Solution:
> >
> > The Sending MTA/MSA must make sure that the "MAIL FROM" in the SMTP
> > envelope MUST always contain the valid local authenticated e-mail
> > address of the of the sender no matter what the sender puts in the
> > "FROM" or "REPLY-TO" fields in the message header. Both RFC 2476 and
> > 2821should be updated to require this since it's currently ambiguous.
> >
> > The Sending MTA Should replace or add, as required, a "SENDER" field to
> > the message header, as stated in RFC 2476, that matched the "MAIL
> > FROM" used in the SMTP envelope.
> >
> > I think that RFC 2476 should be changed to make this a MUST rather than
> > SHOULD requirement.
>
> Getting an RFC through IETF that would change RFC2476 and RFC2821 is
> going to be *real* hard. Getting actual MTAs in the field to comply
> with those changes will take decades.

I would think that SRS would be even harder. RFC 2476 basically states
what I thinking about for the "Sender" field but only for initial
sending of a message and not for forwarding. The "MAIL FROM" only
states an e-mail address and does not state if that address should be
the authenticated one or the one given in the "From" field.

>
> Note that RFC2821 is still a *proposed standard*, it still needs to
> get through the *draft standard* stage before it can supersede RFC821
> as an *internet standard*. RFC2821 was written in Apr 2001, close to
> three years ago. It will be lucky to make it through the final two
> stages this year.

But I think that most e-mail servers are being written to 2821 even
thought is not final.

>
>
> > Win, Win, Win so what am I missing?
>
> I want something done within my lifetime. Your proposed changes to
> RFCs will not be reflected in the real world for a *very* long time.
>

It's really how you read the RFC. There is nothing I read in the RFC
that states what e-mail address should be in the "MAIL FROM" so one
could just start using the authenticated one. In the case of "Sender"
it is stated for some function but not all.


>
> -wayne
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
John Warren
2004-01-19 05:20:16 UTC
Permalink
On 18 Jan 2004 at 23:10, Alex van den Bogaerdt wrote:

> On Sun, Jan 18, 2004 at 08:04:03AM -0800, John Warren wrote:
>
> > Solution:
> >
> > The Sending MTA/MSA must make sure that the "MAIL FROM" in the SMTP
> > envelope MUST always contain the valid local authenticated e-mail
> > address of the of the sender no matter what the sender puts in the
> > "FROM" or "REPLY-TO" fields in the message header. Both RFC 2476 and
> > 2821should be updated to require this since it's currently ambiguous.
> >
> > The Sending MTA Should replace or add, as required, a "SENDER" field to
> > the message header, as stated in RFC 2476, that matched the "MAIL
> > FROM" used in the SMTP envelope.
>
> Just to make sure I understand everything you say, and in the mean
> time asking some questions. Is the following a correct interpretation
> of your message, and can you answer the questions?
>
> User <xyz-***@public.gmane.org> is on the road. He want to send a message
> to <abc-iX3ng/***@public.gmane.org>, and do so from ISP "isp.tld" as user <***@isp.tld>
>
> From pqr-***@public.gmane.org
> Received-by ....
> ...etc...
> From: "user" <xyz-***@public.gmane.org>
> To: "other" <abc-iX3ng/***@public.gmane.org>
> Sender: "user" <pqr-***@public.gmane.org>
> ...etc...
>
> Bounces (if any) will go to pqr-***@public.gmane.org and replies will go to
> xyz-***@public.gmane.org
>

Correct.

> So far so good. Now, this user wants to be able to read his mail.
> At domain.tld, there is a forward setup. "Other" is replying to
> the message and will do so to <xyz-***@public.gmane.org>. domain.tld is
> accepting the message, the .forward is processed and mail is being
> sent out again to <pqr-***@public.gmane.org>.
>
> This will look like:
>
> From xyz-***@public.gmane.org

I don't see why the above line was shown so I will ignore it.

> Received-by ....
> ...etc...
> From: "Other" <other-iX3ng/***@public.gmane.org>
> To: "user" <xyz-***@public.gmane.org>
> Sender: "user" <xyz-***@public.gmane.org>
> ...etc...

No it forwarded would be as follows:

Received-by ....
...etc...
From: "Other" <other-iX3ng/***@public.gmane.org>
To: "user" <xyz-***@public.gmane.org>
Sender: "user" <xyz-***@public.gmane.org>
Resent-from: <xyz-***@public.gmane.org>
Resent-to: <pqr-***@public.gmane.org>
...etc...

The "resent-xxx" would let the mail server know that it was a forward
if there is a bounce so you don't get into a mail loop. See RFC 821 and
2821 3.6.6. Resent fields.


>
> Bounces, if any, will go to <xyz-***@public.gmane.org> and not to <***@other.tld>
> According to some, this is unacceptable.

Why is this unacceptable? The sender "other-iX3ng/***@public.gmane.org" had their
message delivered to the correct address. If the message fails to
forward I don't think that the bounce message has or needs to go back
to the original sender since the message is still in the mailbox and
can be picked up by the user at a later date.

> Consider this:
>
> mail from: <xyz-***@public.gmane.org>
> 250 ...
> rcpt to: <pqr-***@public.gmane.org>
> 5xx mailbox full

Not to be picky but I think that mailbox full would be a 4XX to allow
for a retry. I do get the point though.

>
> At this moment, I think the MTA at "domain.tld" could process the
> bounce to <other-iX3ng/***@public.gmane.org> but this does currently not happen. It
> will go to <xyz-***@public.gmane.org> at the moment.

Which is fine although you want to keep it from getting into a mail
loop. See above.

>
> Another possibility is that the bounce is not generated by "domain.tld"
> but by "isp.tld" and the resulting mail will be introduced like this:
>
> mail from: <>
> rcpt to: <xyz-***@public.gmane.org>
>
> Again, the bounce will be delivered to "xyz", not "other". Even worse,
> many users will forward this bounce again, in stead of storing it
> locally (good enough) or, even better, letting <other-iX3ng/***@public.gmane.org> know.

Again the "Resent-xxx" should keep us from getting into a mail loop.

>
> IMHO "domain.tld" accepted responsability for the message and should
> therefore accept responibility for informing <other-iX3ng/***@public.gmane.org> about
> any delivery problems.

And the message was delivered to the correct user. If that user
forwarded the message to another mail box and it fails I see no reason
that this be sent back to the original sender.

>
> It may be possible no direct communication can or may occur between
> "isp.tld" and "other.tld" (for instance: blacklisting) so sending
> the bounce from "isp.tld" to "other.tld" (as currently would happen
> without SPF) is bad in it self.

Which is another reason that the bounce go back to xyz-***@public.gmane.org and
not other.tld.

>
> Last but not least, "other.tld" does not expect a bounce from "isp.tld"
> thus the bounce may end up inbetween all other spam. Would the bounce
> have been sent by "domain.tld", it has a bigger chance to survive.

I would agree if I accepted that the bounce go back to other.tld which
I don't.

>
> How does your proposal handle bounces? Can you provide an example
> which includes forwarding, via a blind gateway that accepts messages
> without being able to check if it can be delivered?

I'm sorry I don't know what you mean by a "blind gateway". In the case
of a simple forward bounces go back to the forwarding e-mail address
and the "Resent-xxx" should take care of the mail loops.

I should point out here that RFC 821 allowed "Resend-" to be used for a
forward where 2821 does not. I don't understand why that was changed
since there was nothing put in to account for forwarding in 2821.


>
> cheers,
> Alex
> --
> begin sig
> http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
> This message was produced without any <iframe tags
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Za'mbori, Zolta'n
2004-01-19 18:09:32 UTC
Permalink
John Warren wrote:
>>User <xyz-ps1yJmh03mYq8ddH4TG6mw_public.gmane.org> is on the road. He want to send a message
>>to <abc-iX3ng/XTzyZgoHlPtYpdqQ_public.gmane.org>, and do so from ISP "isp.tld" as user <pqr-***@public.gmane.org>

Please do not write valid domain names in examples. Gmane rewrites them
but it does not rewrite 'pqr-***@public.gmane.org' and 'pqr AT isp.tld'.


z2

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Za'mbori, Zolta'n
2004-01-19 18:30:05 UTC
Permalink
Za'mbori, Zolta'n wrote:
> Please do not write valid domain names in examples. Gmane rewrites them
> but it does not rewrite 'pqr-gpJwYPNtiKk_public.gmane.org' and 'pqr AT
> isp.tld'.

I was too optimistic.

z2

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Alex van den Bogaerdt
2004-01-19 22:27:24 UTC
Permalink
On Sun, Jan 18, 2004 at 09:20:16PM -0800, John Warren wrote:

> > This will look like:
> >
> > From xyz-***@public.gmane.org
>
> I don't see why the above line was shown so I will ignore it.

Because we deal with RFC821 stuff here, this line is the result
of "mail from: <...>".


> > Bounces, if any, will go to <xyz-***@public.gmane.org> and not to <***@other.tld>
> > According to some, this is unacceptable.
>
> Why is this unacceptable? The sender "other-iX3ng/***@public.gmane.org" had their
> message delivered to the correct address. If the message fails to
> forward I don't think that the bounce message has or needs to go back
> to the original sender since the message is still in the mailbox and
> can be picked up by the user at a later date.

I happen to agree with you. However, others think the bounce should
go to the original submitter, not the retransmitter. They think this
is _the_ flaw of SPF.

> > Consider this:
> >
> > mail from: <xyz-***@public.gmane.org>
> > 250 ...
> > rcpt to: <pqr-***@public.gmane.org>
> > 5xx mailbox full
>
> Not to be picky but I think that mailbox full would be a 4XX to allow
> for a retry. I do get the point though.

Oops :) Glad you're not too picky but I do appreciate the feedback.
Note to self: check examples next time.

cheers,
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Marc Alaia
2004-01-20 02:25:11 UTC
Permalink
I think that an unreasonably-high priority is being put on terminating the
message before full receipt. The cost-factor may be high now, but once SPF
is more prevalent spammers will not send non-SPF-compliant emails knowing
that they will be simply deleted by the receiving system and therefore doing
it this way will eventually be worthless. I'd rather we do it right in the
beginning than having less perfect system because one group of users sees it
more economically right now....

Marc

-----Original Message-----
From: Alain Knaff [mailto:spf-cKZYRm04eFccDvp3D6m/***@public.gmane.org]
Sent: Saturday, January 17, 2004 10:24 AM
To: spf-discuss-***@public.gmane.org; John Warren; spf-***@v2.listbox.com
Subject: Re: [spf-discuss] Solving the Forwarding Problem for good!!!


begin Saturday 17 January 2004 16:14, John Warren quote:
> SPF tests should always use the "Sender" if there present, not the
> "From".
>
> So the SPF check should work as follows.
>
> If there is a "Sender" use it for checking and ignore the "From" else
> use "From".

The problem with this approach (and others approaches based on header
fields, rather than the envelope) is that it forces the MTA to
actually accept the message, before being able to reject it.

With SPF as it stands now, the mail transaction can already be aborted
before a single byte of message data is transferred.

Alain


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to
http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Meng Weng Wong
2004-01-20 02:57:17 UTC
Permalink
On Mon, Jan 19, 2004 at 09:25:11PM -0500, Marc Alaia wrote:
|
| I think that an unreasonably-high priority is being put on terminating the
| message before full receipt. The cost-factor may be high now, but once SPF
| is more prevalent spammers will not send non-SPF-compliant emails knowing
| that they will be simply deleted by the receiving system and therefore doing
| it this way will eventually be worthless. I'd rather we do it right in the
| beginning than having less perfect system because one group of users sees it
| more economically right now....
|

This could be true. What do people think will happen? Any spammers on
the list please speak up, your input is important. :)

But processing headers is costly. And I don't know if worms and viruses
will care.



-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Julian Mehnle
2004-01-20 10:45:00 UTC
Permalink
Meng Weng Wong [mengwong-GXA0fb4l1L89V/L/***@public.gmane.org] wrote:
> On Mon, Jan 19, 2004 at 09:25:11PM -0500, Marc Alaia wrote:
> > I think that an unreasonably-high priority is being put on
> > terminating the message before full receipt. The cost-factor may be
> > high now, but once SPF is more prevalent spammers will not send
> > non-SPF-compliant emails knowing that they will be simply deleted by
> > the receiving system and therefore doing it this way will eventually
> > be worthless. I'd rather we do it right in the beginning than having
> > less perfect system because one group of users sees it more
> > economically right now....
>
> This could be true. What do people think will happen? Any spammers on
> the list please speak up, your input is important. :)
>
> But processing headers is costly. And I don't know if worms and
> viruses will care.

"Once SPF is more prevalent", who will write virm that forge the sender address, knowing that the messages will be rejected/deleted automatically(!) anyway?

I don't quite understand the "processing headers is costly" argument, though. Why would SPF need to look at the headers in the first place? If I understand John Warren's proposal correctly, it essentially just says that forwarding/bounce information should be put into the "Sender:" header field instead of using a complicated rewriting scheme for the envelope sender. The core functionality of SPF, i.e. determining if the envelope sender domain is "valid", isn't touched by that change, is it?

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.orgx.com
Alex van den Bogaerdt
2004-01-20 22:20:56 UTC
Permalink
On Tue, Jan 20, 2004 at 11:45:00AM +0100, Julian Mehnle wrote:

> I don't quite understand the "processing headers is costly" argument

In order to be able to process the headers, they have to be received
in the first place.

The difference is that in one kind of proposal one is able to stop
a mail right after one of the RFC 821 type of commands (upto and
including "DAtA") whereas in the other kind of proposal one has to
receive the attached message (which could be huge!), process it and
delete it.

The difference is not in processing itself (at least: no significant
difference) but in transfering the body or not.

HTH
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Julian Mehnle
2004-01-21 01:39:55 UTC
Permalink
Alex van den Bogaerdt [***@ergens.op.het.net] wrote:
> On Tue, Jan 20, 2004 at 11:45:00AM +0100, Julian Mehnle wrote:
> > I don't quite understand the "processing headers is costly" argument
>
> In order to be able to process the headers, they have to be received in
> the first place.
>
> The difference is that in one kind of proposal one is able to stop
> a mail right after one of the RFC 821 type of commands (upto and
> including "DAtA") whereas in the other kind of proposal one has to
> receive the attached message (which could be huge!), process it and
> delete it.
>
> The difference is not in processing itself (at least: no significant
> difference) but in transfering the body or not.

That part I *do* understand. What I do *not* understand is why reading (and therefore downloading) the header lines is required in the first place?

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Alex van den Bogaerdt
2004-01-21 12:15:57 UTC
Permalink
On Wed, Jan 21, 2004 at 02:39:55AM +0100, Julian Mehnle wrote:

>
> That part I *do* understand. What I do *not* understand is why reading (and therefore downloading) the header lines is required in the first place?

In another message, please find and read this part:

> > If there is a "Sender" use it for checking and ignore the "From" else
> > use "From".

> The problem with this approach (and others approaches based on header
> fields, rather than the envelope) is that it forces the MTA to
> actually accept the message, before being able to reject it.

> With SPF as it stands now, the mail transaction can already be aborted
> before a single byte of message data is transferred.

You were responding to this sub-thread so logically I assumed you were
talking about RFC822 "Sender: " and "From: " vs. RFC821 "mail from:"

Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-21 15:15:24 UTC
Permalink
On Wed, Jan 21, 2004 at 01:15:57PM +0100, Alex van den Bogaerdt wrote:
|
| > The problem with this approach (and others approaches based on header
| > fields, rather than the envelope) is that it forces the MTA to
| > actually accept the message, before being able to reject it.
|

to be precise, it doesn't require the MTA to accept it. it requires the
MTA to transfer it, which is a bandwidth cost. in theory the MTA can
still reject it at "." time.

practically speaking, i hear there are a number of MTAs that treat all
"." time rejections as tempfail, and will retry.


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Alex van den Bogaerdt
2004-01-21 16:53:46 UTC
Permalink
On Wed, Jan 21, 2004 at 10:15:24AM -0500, Meng Weng Wong wrote:
> On Wed, Jan 21, 2004 at 01:15:57PM +0100, Alex van den Bogaerdt wrote:
> |
> | > The problem with this approach (and others approaches based on header
> | > fields, rather than the envelope) is that it forces the MTA to
> | > actually accept the message, before being able to reject it.
> |
>
> to be precise, it doesn't require the MTA to accept it. it requires the
> MTA to transfer it, which is a bandwidth cost. in theory the MTA can
> still reject it at "." time.

Indeed. One of these days I'm going to learn how to write what I think.

> practically speaking, i hear there are a number of MTAs that treat all
> "." time rejections as tempfail, and will retry.

This I did not now, but then again I'm not surprised either.

cheers,
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-21 18:04:31 UTC
Permalink
On Wed, Jan 21, 2004 at 05:53:46PM +0100, Alex van den Bogaerdt wrote:
| >
| > to be precise, it doesn't require the MTA to accept it. it requires the
| > MTA to transfer it, which is a bandwidth cost. in theory the MTA can
| > still reject it at "." time.
|
| Indeed. One of these days I'm going to learn how to write what I think.
|

Could ISPs please weigh in on whether your email/spam bandwidth cost is
signicant or not, thanks.



-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
r***@tclme.org
2004-01-21 18:36:00 UTC
Permalink
On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong asserted:
> On Wed, Jan 21, 2004 at 05:53:46PM +0100, Alex van den Bogaerdt wrote:
> | >
> | > to be precise, it doesn't require the MTA to accept it. it requires the
> | > MTA to transfer it, which is a bandwidth cost. in theory the MTA can
> | > still reject it at "." time.
> |
> | Indeed. One of these days I'm going to learn how to write what I think.
> |
>
> Could ISPs please weigh in on whether your email/spam bandwidth cost is
> signicant or not, thanks.
>
>

Uhh, yeah...

Imagine having a big parking lot of your very own. Now (we'll call him Mr. Wallace) comes along and points out that even though you own the parking lot, the law clearly states that unless you catch him as he attempts to park, you may only politely ask the he not place his hoard of cars in your lot. Of course, being a law abiding citizen, you shoot him where he stands and... (Oh wait, I was dreaming again.) When your lot fills with cars brought in by Mr. Wallace, you find yourself with a decision: Do you begin diligent monitoring of entries into your lot (erect a gate?), purchase additional land to build more lots that may suffer the same fate as the one you own now, or do you just give up and resign to your fate as a victim because you know that you'll never succeed in changing the political lanscape by yourself?

--

Bob Greene
Public key available at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC9C7841C
Or, you can just pull my finger

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Andy Lester
2004-01-21 18:36:10 UTC
Permalink
> > Could ISPs please weigh in on whether your email/spam bandwidth cost is
> > signicant or not, thanks.
>
> Uhh, yeah...
>
> Imagine having a big parking lot of your very own.

I think Meng understands the allegory. the question is, how much does
the bandwidth cost affect your bottom line?

xoa

--
Andy Lester => ***@petdance.com => www.petdance.com => AIM:petdance

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Sam Norris
2004-01-21 18:39:26 UTC
Permalink
Bandwidth is 90% of the cost almost ... space is easy to find, its the
bandwidth that you pay for ... at least in a decent data center where
everyones not piped thru a T1.

Sam

----- Original Message -----
From: "Andy Lester" <***@petdance.com>
To: <spf-***@v2.listbox.com>
Sent: Wednesday, January 21, 2004 10:36 AM
Subject: Re: [spf-discuss] Calling ISPs: does bandwidth matter to your
bottom line?


> > > Could ISPs please weigh in on whether your email/spam bandwidth cost
is
> > > signicant or not, thanks.
> >
> > Uhh, yeah...
> >
> > Imagine having a big parking lot of your very own.
>
> I think Meng understands the allegory. the question is, how much does
> the bandwidth cost affect your bottom line?
>
> xoa
>
> --
> Andy Lester => ***@petdance.com => www.petdance.com => AIM:petdance
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your
subscription,
> please go to
http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Hans Dieter Pearcey
2004-01-21 18:51:41 UTC
Permalink
On Wed, Jan 21, 2004 at 10:39:26AM -0800, Sam Norris wrote:
> Bandwidth is 90% of the cost almost ... space is easy to find, its the
> bandwidth that you pay for ... at least in a decent data center where
> everyones not piped thru a T1.

The question is not whether or not bandwidth costs money, but whether --
for an ISP, which presumably has a whole lot of non-email traffic,
which you KNOW because your ssh session always slows down when the guy
in the next apartment over starts using Kazaa -- the cost of the traffic
used by spam is a significant part of the total bandwidth cost.

For us, spam costs a high percentage of our bandwidth, because most of
said bandwith is used for email. I don't know if that's true for a
typical ISP, with home users and so on.

hdp.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
r***@tclme.org
2004-01-21 19:03:36 UTC
Permalink
On Wed, Jan 21, 2004 at 12:36:10PM -0600, Andy Lester asserted:
> > > Could ISPs please weigh in on whether your email/spam bandwidth cost is
> > > signicant or not, thanks.
> >
> > Uhh, yeah...
> >
> > Imagine having a big parking lot of your very own.
>
> I think Meng understands the allegory. the question is, how much does
> the bandwidth cost affect your bottom line?
>
> xoa
>

I realize that he understands. I was incomplete in my tale. Thank you for pointing it out.

In my case, bandwidth similar to the parking lot, its all I sell. Its also my main expense. Support (phones & staff) would come a close second and those generally increase as my available bandwidth decreases.

--

Bob Greene
Public key available at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC9C7841C
Or, you can just pull my finger

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-21 20:53:15 UTC
Permalink
On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
| Could ISPs please weigh in on whether your email/spam bandwidth cost is
| signicant or not, thanks.

The decision at hand is:

How important is it to reject forgeries before DATA vs after DATA at "." time.

If it is important, that's an argument in favour of the envelope sender,
otherwise the argument allows use of headers.


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
John Capo
2004-01-21 21:04:32 UTC
Permalink
Quoting Meng Weng Wong (***@dumbo.pobox.com):
> On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
> | Could ISPs please weigh in on whether your email/spam bandwidth cost is
> | signicant or not, thanks.
>
> The decision at hand is:
>
> How important is it to reject forgeries before DATA vs after DATA at "." time.

Its quite easy to add the hooks needed to reject before DATA to a
lot of MTAs, especially Postfix. Rejecting after DATA is much more
complicated. In my case a system redesign.

If I accepted all of the junk that I reject at the door, my bandwidth
costs would increase about 50%.

>
> If it is important, that's an argument in favour of the envelope sender,
> otherwise the argument allows use of headers.
>
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Larry Smith
2004-01-21 21:30:52 UTC
Permalink
On Wednesday 21 January 2004 15:04, John Capo wrote:
> Quoting Meng Weng Wong (***@dumbo.pobox.com):
> > On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
>> |
> > | Could ISPs please weigh in on whether your email/spam bandwidth cost is
> > | signicant or not, thanks.
> >
> > The decision at hand is:
> >
> > How important is it to reject forgeries before DATA vs after DATA at "."
> > time.
>
> Its quite easy to add the hooks needed to reject before DATA to a
> lot of MTAs, especially Postfix. Rejecting after DATA is much more
> complicated. In my case a system redesign.
>
> If I accepted all of the junk that I reject at the door, my bandwidth
> costs would increase about 50%.
>
> > If it is important, that's an argument in favour of the envelope sender,
> > otherwise the argument allows use of headers.
> >

Weighing in on the "light-weight" scale of things (I am small and don't
weight much)... I would have to agree with Bob and John specifically in the
context that as an "ISP" - I sell "access" which corelates almost directly to
"bandwidth". The more spam, the less "bandwidth" available for other (more
legitimate) uses; the more spam, the more I have to tweak and work with my
mail servers and routers and other "backbone" support issues -and the more
time me and my support staff spend talking, working with, consoling, etc,
etc. customers who really don't understand why (they get so much "junk"
mail)...

On the one side I have customers that simply stop reading their mail because
there is just too much "junk" (from their perspective) to deal with; I also
have (ex)customers who cancelled their service just because they figured
changing service (read that email accounts) would "help" at least in the
short term.

The bottom line to this being - the sooner (eg before the helo if possible,
but definitely before the data if _at_all_ possible) that I can "reject" a
message/connection and get off the line so-to-speak; the sooner my server can
go on to something else productive. Regardless of who or what you support,
there are at some point a finite level of resources available - even
"super-servers" max out at some value of X number of connections (be it smtp,
nntp, www, or whatever). The "big" guys can load-balance, split-horizon,
share-resources and so forth - but we are all seeing the same result -
resources being consumed almost as fast as we upgrade.

My vote is for determination _prior_ to the data statement....

--
Larry Smith
SysAd ECSIS.NET
***@ecsis.net


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Rob Kaper
2004-01-21 21:21:23 UTC
Permalink
On Wed, Jan 21, 2004 at 03:53:15PM -0500, Meng Weng Wong wrote:
> If it is important, that's an argument in favour of the envelope sender,
> otherwise the argument allows use of headers.

Blocking based on headers doesn't exclude the option to block on envelope
sender. If you'd ask me, *both* should not be forged, thus *both* should be
protected by SPF.

Rob
--
Rob Kaper | "We should limit the burden of government on this economy."
***@capsi.com |
www.capsi.com | -- President George W. Bush, State of the Union, 1/20/2004

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
William Astle
2004-01-21 14:49:32 UTC
Permalink
On Wed, 21 Jan 2004, Meng Weng Wong wrote:

> The decision at hand is:
>
> How important is it to reject forgeries before DATA vs after DATA at "."
> time.
>
> If it is important, that's an argument in favour of the envelope sender,
> otherwise the argument allows use of headers.

Wearing my ISP hat, I say it is important to reject before DATA. If I can
reject based on the envelope, I don't have to deal with the inbound
bandwidth of the message being delivered.

--
William Astle
finger ***@l-w.net for further information

Geek Code V3.12: GCS/M/S d- s+:+ !a C++ UL++++$ P++ L+++ !E W++ !N w--- !O
!M PS PE V-- Y+ PGP t+@ 5++ X !R tv+@ b+++@ !DI D? G e++ h+ y?

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
r***@tclme.org
2004-01-21 23:13:08 UTC
Permalink
On Wed, Jan 21, 2004 at 07:49:32AM -0700, William Astle asserted:
> On Wed, 21 Jan 2004, Meng Weng Wong wrote:
>
> > The decision at hand is:
> >
> > How important is it to reject forgeries before DATA vs after DATA at "."
> > time.
> >
> > If it is important, that's an argument in favour of the envelope sender,
> > otherwise the argument allows use of headers.
>
> Wearing my ISP hat, I say it is important to reject before DATA. If I can
> reject based on the envelope, I don't have to deal with the inbound
> bandwidth of the message being delivered.
>

Much more important to handle it before data. As has been stated before, at that point, the disposition is final.

--

Bob Greene
Public key available at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC9C7841C
Or, you can just pull my finger

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-23 15:58:34 UTC
Permalink
On Wed, 2004-01-21 at 15:53, Meng Weng Wong wrote:
>
> The decision at hand is:
>
> How important is it to reject forgeries before DATA vs after DATA at "." time.
>

Okay, I'm not an ISP, (IANAISP--or does IANANISP look better), but:

Why not both?

First do the envelope-based tests in the current specification, then do
the currently-debated tests on the message body.

First pass: I would suspect that most current mail forgeries could be
rejected before DATA. For these rejected emails, no additional
bandwidth, cpu, or network latencies need occur.

Second pass: For connections that are not rejected at this point, the
additional tests in debate (and continually submitted) could be
performed at "." time. For this second pass at testing, you'll have to
incur the following costs:

1. The bandwidth cost of receiving the full email. This is not an
issue, because if you *didn't* do this second test, you'll have
to get the body of the mail just the same as if you did.

2. CPU usage. This is CPU usage that would otherwise likely occur
at some other point anyway, (ISP-level filtering outside of the
MX machines, or client-level filtering). The issue is not so much
whether it be done, but where it should be done.

My bias is that I'd be rather put-off by an ISP pushing
deterministic anti-forgery tests onto me because they didn't want to
spend the CPU to do it themselves at the get-go and generate a
proper reject when the opportunity existed.

3. Network latency. Some of the proposed tests could require
additional (also proposed) spf-type DNS queries. This means
additional MTA-processes out there waiting on DNS responses.

4. The nuisance of technical incompatibilities. MTA's that don't
understand reject-after-data have to be worked around in some
manner.

The problem of MTA's that consider a permanent failure reject at
this point to be a transient failure instead is the one that comes
to mind.

How to handle this particular nuisance would unquestionably be
an item for debate here.

IMO, this is broken behavior that should be fixed; meaning that I
would prefer rather harsh responses that mean people tend to
immediately recognize that their systems are broken and quickly fix
them, rather than have the proposed-additions-to-the-standard
modified in ways that may inadvertently break it or allow more
forgeries to get through. (Granted, this non-accepting behavior
is very un-RFC-like.)

(I'm partial to some sort of automated black-holing of
folks sending forged emails with broken software--the receiving
MTA keeping track of triples of sending-ip, envelope data,
checksums of body data for a few days, and after receiving some
number of duplicate sending-ip and envelope body
checksums, "black-holing" that envelope-sender/ip combination for
a few days by sending permanent rejects to all connections, with
appropriate error messages. However, this is an
off-the-top-of-my-head response, so it probably contains a lot of
fundamental problems.)

Also, there *could* be an addition to the spec so that
a domain can publish in their SPF records the fact that their MTA
or mail sending programs (I hesitate to say MUAs as that sounds like
it doesn't include programs that include their own code to send out
emails themselves directly) are broken in this way. (I'd rather
there not be such an addition but that the would-be publisher fixes
his software first.)

5. MTA's that require a lot of rewriting to test the body of emails
this way will be locked out of being able to quickly run these
tests.

Well, I suppose those MTA's can just ignore the full-body
tests, or push them off to other machines or the users.

Until the MTA's are rewritten, or they switch to another, these
people will receive more forgeries than people with more adaptable
MTA's, but there's no sense in everyone being held back because a
few find life difficult with their current software.

Note this is also in keeping with the notion of making every step
of SPF deployment advantageous to all parties: These people will
gain advantages by publishing SPF records, by filtering on the
envelope, and by filtering on the body.

In any event, #4 above is the only argument I can see as truly valid for
rejecting (haha) the idea of "."-time rejects, and it's not enough to
sway me really, (for what that's worth.)

As an aside, IMO sending a reject is friendly behavior to the
inadvertent forger. Delaying these tests and thus pushing them off to
the end-users who potentially run their own anti-spam and forgery
testing code means that either these users:

1. Don't run such programs and get more forgeries and forged spam, or

2. Do run such programs which either:

A. send bounces themselves, (unlikely, but still scary to think
about),
B. delete the emails unread, (meaning the hapless inadvertent-
forger doesn't realize he has a problem and his mails aren't
being read), or
C. The mails are marked as "probably-a-forgery", and the end
user gets a bad taste in his mouth about the whole procedure,
as spf doesn't seem to be working as advertised. (Remember,
the end user will *NOT* see the reject-before data emails,
rather he'll be judging SPF's effectiveness mostly on the basis
of the forgeries that do get through. I'm wondering if from
the end-user point of view, if it will look like these are
emails that should never have even made it *to* his spam
filter. As an aside, I guess this means that ISP's will
want to publicize to their users the average number of spams
per user blocked every week by SPF!)

To me it just makes sense to do tests at both places.

In any event, I'm guessing that spf2 will probably contain functionality
that would require tests of the whole message anyway. If that's
generally considered to be the case, then to me it makes sense to have
the spec to go ahead and force the issue now instead of later.

So again, to me it just makes sense to do tests at both places.

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-23 16:23:14 UTC
Permalink
On Fri, 23 Jan 2004, Mark Shewmaker wrote:

: > The decision at hand is:
: >
: > How important is it to reject forgeries before DATA vs after DATA at "." time.
:
: Okay, I'm not an ISP, (IANAISP--or does IANANISP look better), but:
:
: Why not both?

Because DATA can be arbitrarily large, and requires consumption of the
entire message. Most MTAs will choke if the DATA phase terminates abruptly
without consuming all data, and will turn around and send the message again.
This is a bandwidth waste problem. There's another, legal, problem with
rejecting after DATA; see below for more explanation.

Thus, reliable, well-deployed mail rejections must happen before DATA.
Sending a 5xx after DATA is "filtering", not "rejection", because the SMTP
agent had to accept the data before sending the 5xx!

Sure, go ahead and provide *suggestions* on how to implement filtering using
SPF and DATA-phase headers. PLEASE DO NOT MAKE THIS A "MUST" OR "SHOULD" IN
A RFC-TARGETED SPEC. Doing so will seriously degrade the feasibility of SPF
(even if dubbed "v2") implementation in most large ISPs.

<decloak style="frosted-glass">
I happen to know the internal folks of one such large ISP, and why they will
not implement SPF if it requires body checks.
</decloak>

: My bias is that I'd be rather put-off by an ISP pushing
: deterministic anti-forgery tests onto me because they didn't want to
: spend the CPU to do it themselves at the get-go and generate a
: proper reject when the opportunity existed.

ISPs can also get into legal trouble in some jurisdictions because only
rejection at RCPT TO time can provide each end-user the ability to choose
whether or not to use a given type of filter. Rejection at both MAIL FROM
and DATA have the effect of rejecting all recipients in a multi-recipient
mail.

: In any event, I'm guessing that spf2 will probably contain functionality
: that would require tests of the whole message anyway.

I sure hope not. I might as well pack up on the SPF idea and go home if it
turns into just another "filtering" scheme, because it'll end up being no
more useful than all the other filtering I have to do today.

SPF needs to provide determinism at the protocol level -- before a whole
message is accepted.

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-23 19:17:19 UTC
Permalink
On Fri, 2004-01-23 at 11:23, tv+***@duh.org wrote:
> On Fri, 23 Jan 2004, Mark Shewmaker wrote:
>
> : > The decision at hand is:
> : >
> : > How important is it to reject forgeries before DATA vs after DATA at "." time.
> :
> : Okay, I'm not an ISP, (IANAISP--or does IANANISP look better), but:
> :
> : Why not both?
>
> Because DATA can be arbitrarily large, and requires consumption of the
> entire message. Most MTAs will choke if the DATA phase terminates abruptly
> without consuming all data, and will turn around and send the message again.
> This is a bandwidth waste problem.

Okay.

So if you keep track of the triples I mentioned, (which requires a lot
of work of course), and you decide to do the black-holing of envelopes
corresponding to resent messages on the second delivery attempt, then
the bandwidth increase you'll see will be due to a doubling of email
messages in which:

1. The before-DATA test passed.
2. The mail is still a forgery according to debated-to-be-added SPF
heuristics.
3. The sender's machine chokes on rejects at "." time. (You say most
MTA's are like this.)

I suppose one question is what percentage of all received emails fit all
three requirements above?

A second question would be what percentage fit just 1 and 2? (That is,
how many emails *pass* spf envelope checks but would *fail* the proposed
body header checks?)

A third question is how would we interpret any percentage?

If the percentage is considered low, then there actually wouldn't be
much of a bandwidth problem. Spammers would be dissuaded from trying to
adapt-around spf restrictions on forgeries because spf has built-in
defenses to that and it would get them nowhere. (Sending
non-forged-envelopes from probably-blackholed IPs with
would-fail-spf-check body headers won't get them far in a world where
spf is being more and more widely deployed.) So, I expect that the
result would be that this percentage would stay low, and get lower as
MTAs are very slowly updated.

If the percentage is high, then that means that SPF isn't going to be
very effective at preventing forgeries from just envelope data in the
first place, and we'd really need to add in header checks for it to be
at all workable.

(As an aside, a spammer would probably want to send email from a MTA
that handles at-"."-rejects properly, as otherwise they'll be wastfully
resending to places they'll never get their mail to anyway. Strange to
think of something as possibly being useful because of its advantage for
spammers though!)

> There's another, legal, problem with
> rejecting after DATA; see below for more explanation.
>
> Thus, reliable, well-deployed mail rejections must happen before DATA.
> Sending a 5xx after DATA is "filtering", not "rejection", because the SMTP
> agent had to accept the data before sending the 5xx!

Given the possibility of saving the spf-rejected triples, I don't see
that that argument follows.

> Sure, go ahead and provide *suggestions* on how to implement filtering using
> SPF and DATA-phase headers. PLEASE DO NOT MAKE THIS A "MUST" OR "SHOULD" IN
> A RFC-TARGETED SPEC. Doing so will seriously degrade the feasibility of SPF
> (even if dubbed "v2") implementation in most large ISPs.

The draft in progress says what you MUST do if you decide to have an SPF
client (that checks on the basis of envelope data and DNS queries.)

I would suggest that the draft in progress could reasonably specify what
you MUST do if your SPF client is also to do spf-tests on the body.

But I would not suggest that the draft say that clients that do envelope
checking, (the only kind currently listed), MUST also do any other sort
of tests as well. (With obvious exceptions for changes that may become
necessary for v2 that we don't currently don't realize are or will be
important.)

I believe that addresses this concern of yours.

However, I do have a question: What would you think of a requirement
that your authorized outgoing MTA machines according to your published
SPF records SHOULD properly handle rejects-at-"."?

> <decloak style="frosted-glass">
> I happen to know the internal folks of one such large ISP, and why they will
> not implement SPF if it requires body checks.
> </decloak>

This is the Mystery Supporter argument in another form.

In any event, if body checks are really to everyones advantage, then
those who don't do them will simply be at a disadvantage. If they are
far more trouble that they're worth, then people simply won't adopt that
section. Just like spf as a whole really.

I believe they'll be useful, and I want to deploy something like what's
been discussed at some point. Before then of course, you may have
persuaded me of the error of my ways.

> ISPs can also get into legal trouble in some jurisdictions because only
> rejection at RCPT TO time can provide each end-user the ability to choose
> whether or not to use a given type of filter. Rejection at both MAIL FROM
> and DATA have the effect of rejecting all recipients in a multi-recipient
> mail.

Thanks. I didn't think about that difference.

However, you can still get partway there imperfectly by simply only
doing a reject at "." if all end-users have chosen to use SPF body
"filtering", as you call it.

If less than all have done so, the mail can be accepted and headers
added for any users who did.

(There is a privacy issue here in that if you selected this filtering
and received an email like that, you would know that at least one of the
other recipients didn't select the same filtering. If that's not a
problem that can be solved contractually, then that ISP could simply
merely add headers for all filtered-at-body emails. The proposal would
have to account for this potential snag.)

> : In any event, I'm guessing that spf2 will probably contain functionality
> : that would require tests of the whole message anyway.
>
> I sure hope not. I might as well pack up on the SPF idea and go home if it
> turns into just another "filtering" scheme, because it'll end up being no
> more useful than all the other filtering I have to do today.

That's not a valid comparison. The normal current filtering schemes
can't do sender-permitted checks as spf can.

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-23 20:53:20 UTC
Permalink
On Fri, 23 Jan 2004, Mark Shewmaker wrote:

: > Because DATA can be arbitrarily large, and requires consumption of the
: > entire message. Most MTAs will choke if the DATA phase terminates abruptly
: > without consuming all data, and will turn around and send the message again.
: > This is a bandwidth waste problem.

: So if you keep track of the triples I mentioned,

Keep track of what triples? SPF's purpose is to verify that the sender of a
mail is plausible--nothing more.

: 2. The mail is still a forgery according to debated-to-be-added SPF
: heuristics.

Stopping forgery at the MAIL FROM sender level is pretty cut and dry. I
don't see how you could then additionally consider the mail to be a
"forgery". Exactly what additional information are you trying to check!?

: 3. The sender's machine chokes on rejects at "." time. (You say most
: MTA's are like this.)

I didn't mean that. What I said (explained here in further detail) is that
rejection at DATA requires consuming the whole message, whereas dropping the
TCP connection is equivalent to a transient failure that will cause the
remote MTA to retry. So any check that is at DATA is inherently more
expensive by several orders of magnitude, as the entire data stream must be
received before the 5xx can be sent.

: > Thus, reliable, well-deployed mail rejections must happen before DATA.
: > Sending a 5xx after DATA is "filtering", not "rejection", because the SMTP
: > agent had to accept the data before sending the 5xx!
:
: Given the possibility of saving the spf-rejected triples, I don't see
: that that argument follows.

Huh? The only difference between reject-after-DATA and a filter program is
the 5xx result code. It doesn't matter what you do with the data after that
point; it's been consumed.

And we all know that common forgers don't give a damn about 5xx results and
will keep on sending anyway. So such reject-after-DATA collapses right back
to filtering.

: But I would not suggest that the draft say that clients that do envelope
: checking, (the only kind currently listed), MUST also do any other sort
: of tests as well. (With obvious exceptions for changes that may become
: necessary for v2 that we don't currently don't realize are or will be
: important.)
:
: I believe that addresses this concern of yours.

No. I'd like to see the DATA-phase loophole closed now, before we fall into
that trap. The point of SPF was to make use of SMTP transaction information
to provide a *lightweight* authentication mechanism.

Adding a bunch of DATA phase check possibilities to the core spec encourages
too many mail senders (you know, the ones who create the SPF records in DNS)
to be lazy. This defeats the lightweight check purpose of SPF, which I
believe will lead to too many "unknown" results in the wild due to low
adoption rate.

Please, before we fall into the DATA-phase trap, put it in an extension
document that is not attached to the core SPF spec.

: This is the Mystery Supporter argument in another form.

I have to stay cloaked on mailing lists for contractual reasons, sorry. I
can talk more about that offlist only. Let's just say here that my employer
is not based in the state of Washington, and its core products are
bandwidth, not software. I also only use XML for exporting or importing
data, and rarely that if I can avoid it. 8-)

: In any event, if body checks are really to everyones advantage,

You've never spent much time in anti-spam mailing lists, I presume. It's an
established fact that body checks are *bad* and something we want to
eliminate as much as possible, not integrate further.

Join SPAM-***@PEACH.EASE.LSOFT.COM and ***@ABUSE.NET. There's lots of
folks there who can educate you with statistics relating to the high expense
of rejecting at DATA rather than MAIL FROM or RCPT TO. I've been a regular
on both for several years.

: However, you can still get partway there imperfectly by simply only
: doing a reject at "." if all end-users have chosen to use SPF body
: "filtering", as you call it.

Not likely to happen. Many customers want to receive everything unfiltered
just to avoid even the narrowest possibility of a false positive.

Besides, this would only be useful in some cases. There's far more
effective after-DATA checks that attempt to fix the Spam Problem, which
already fix the forgery problem as a side effect (SpamAssassin is a
semi-decent example). We want something that happens *earlier*.

: If less than all have done so, the mail can be accepted and headers
: added for any users who did.

But then the recipients who wanted to reject never got the chance to do so.

: > I sure hope not. I might as well pack up on the SPF idea and go home if it
: > turns into just another "filtering" scheme, because it'll end up being no
: > more useful than all the other filtering I have to do today.
:
: That's not a valid comparison. The normal current filtering schemes
: can't do sender-permitted checks as spf can.

Actually, there's a motherlode of "mail for foo.dom.ain comes from these
networks" data out there, particularly for such folks as AOL. Filtering
software can already make use of this (SpamAssassin does).

My point, however, is that SPF-checking after DATA is no better at stopping
the *entry* of forgeries into the system than all kinds of other heuristics
on the message header and body which are already done. Doing SPF-on-body
just adds another sieve layer to the already too big body filter mix.

I honestly believe that encouraging implementation of SMTP protocol level
changes (such as SRS) would do much, much more than body checks to make SPF
not just feasible, but very convenient. Even new ESMTP extensions are
probably more likely to be widely adopted than specialized body checks.

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-24 01:11:04 UTC
Permalink
Moot points now that Meng refroze the v1 spec, but for what it's worth:

On Fri, 2004-01-23 at 15:53, tv+***@duh.org wrote:
> On Fri, 23 Jan 2004, Mark Shewmaker wrote:
> : So if you keep track of the triples I mentioned,
>
> Keep track of what triples? SPF's purpose is to verify that the sender of a
> mail is plausible--nothing more.

The ones I mentioned in the first email of mine you responded to today,
keeping track of passed-envelope-check but failed-body-check forgeries,
to prevent infinite retries from buggy sending MTAs.

> : 2. The mail is still a forgery according to debated-to-be-added SPF
> : heuristics.
>
> Stopping forgery at the MAIL FROM sender level is pretty cut and dry. I
> don't see how you could then additionally consider the mail to be a
> "forgery". Exactly what additional information are you trying to check!?

The tests discussed elsewhere on this list in the past week or so
discussing the possibility of neither header body Sender: nor From:
having anything to do with the MAIL FROM sender.

Granted, it's harder for a Spammer to forge emails that way, as they'll
be sending from easily-blackholable IPs, but it was a concern brought up
and discussed a good bit elsewhere on the list.

> : 3. The sender's machine chokes on rejects at "." time. (You say most
> : MTA's are like this.)
>
> I didn't mean that. What I said (explained here in further detail) is that
> rejection at DATA requires consuming the whole message, whereas dropping the
> TCP connection is equivalent to a transient failure that will cause the
> remote MTA to retry.

But my understanding was, especially given other emails on this list
stating this in no certain terms, was that if an MTA considered that a
transient failure that that was a bug in that MTA--there was never any
debate about whether that was a bug, only concern that buggy MTAs were
out there.

> So any check that is at DATA is inherently more
> expensive by several orders of magnitude, as the entire data stream must be
> received before the 5xx can be sent.

Again, at that point you have no choice but to receive that entire data
stream anyway, because you've *already done* the currently-spec'd
envelope spf check, that email passed them, and so you decided not to
reject before DATA for that email.

The greater expense would come not from larger body data streams but
because on the very small percentage of emails that make it past your
envelope tests but not the simple spf body tests: You'll have to
workaround buggy MTA's by keeping track of these past failures to reject
at envelope the second time around, or use some other expensive
workaround.

(I was surprised you didn't strenuously object to that on the first
email, as it would mean you'd have to set up a lightweight database that
your MTA would query at envelope-test time.)

> No. I'd like to see the DATA-phase loophole closed now, before we fall into
> that trap. The point of SPF was to make use of SMTP transaction information
> to provide a *lightweight* authentication mechanism.
>
> Adding a bunch of DATA phase check possibilities to the core spec encourages
> too many mail senders (you know, the ones who create the SPF records in DNS)
> to be lazy. This defeats the lightweight check purpose of SPF, which I
> believe will lead to too many "unknown" results in the wild due to low
> adoption rate.

Honestly that doesn't make sense. I really don't follow the logic and I
don't understand what you mean by being lazy: I can't imagine that
anyone would ever want to skimp on the envelope checks for one, and the
header checks discussed elsewhere are about as clear as the envelope
ones.

As far as mail senders--I don't see how this means they'd want to change
their published SPF records either.

I simply don't follow. What would being lazy entail?

> Please, before we fall into the DATA-phase trap, put it in an extension
> document that is not attached to the core SPF spec.

Uhm, that's effectively the same thing I'm suggesting--I really can't
see the difference between a separate extension and mention of how to do
an additional check that people will surely be doing.

> : This is the Mystery Supporter argument in another form.
>
> I have to stay cloaked on mailing lists for contractual reasons, sorry. I
> can talk more about that offlist only. Let's just say here that my employer
> is not based in the state of Washington, and its core products are
> bandwidth, not software. I also only use XML for exporting or importing
> data, and rarely that if I can avoid it. 8-)

I understand, and I wasn't suggesting you were the Mystery Supporter,
nor that you wanted to throw XML everywhere.

(However it would make for a great April Fool's day SPF V2 spec to
rewrite everything possible in terms of XML, even though that wouldn't
be funny to anyone outside this list.)


> : In any event, if body checks are really to everyones advantage,
>
> You've never spent much time in anti-spam mailing lists, I presume. It's an
> established fact that body checks are *bad* and something we want to
> eliminate as much as possible, not integrate further.
>
> Join SPAM-***@PEACH.EASE.LSOFT.COM and ***@ABUSE.NET. There's lots of
> folks there who can educate you with statistics relating to the high expense
> of rejecting at DATA rather than MAIL FROM or RCPT TO. I've been a regular
> on both for several years.

I'll look at the archives.

> : However, you can still get partway there imperfectly by simply only
> : doing a reject at "." if all end-users have chosen to use SPF body
> : "filtering", as you call it.
>
> Not likely to happen. Many customers want to receive everything unfiltered
> just to avoid even the narrowest possibility of a false positive.

That's fine then. Just don't handcuff the rest of us. :-)

> Besides, this would only be useful in some cases. There's far more
> effective after-DATA checks that attempt to fix the Spam Problem, which
> already fix the forgery problem as a side effect (SpamAssassin is a
> semi-decent example). We want something that happens *earlier*.

I've never, ever objected to the earlier checks.

> : If less than all have done so, the mail can be accepted and headers
> : added for any users who did.
>
> But then the recipients who wanted to reject never got the chance to do so.

You pointed out that that's impossible to do no matter what, and I
understand and agree, so there's no sense worrying about the cases that
can't be handled.

> : > I sure hope not. I might as well pack up on the SPF idea and go home if it
> : > turns into just another "filtering" scheme, because it'll end up being no
> : > more useful than all the other filtering I have to do today.
> :
> : That's not a valid comparison. The normal current filtering schemes
> : can't do sender-permitted checks as spf can.
>
> Actually, there's a motherlode of "mail for foo.dom.ain comes from these
> networks" data out there, particularly for such folks as AOL. Filtering
> software can already make use of this (SpamAssassin does).

Without the possibility of doing rejects and without as much hard data
on small domains--but SPF will fortunately change the latter.

> My point, however, is that SPF-checking after DATA is no better at stopping
> the *entry* of forgeries into the system than all kinds of other heuristics
> on the message header and body which are already done. Doing SPF-on-body
> just adds another sieve layer to the already too big body filter mix.
>
> I honestly believe that encouraging implementation of SMTP protocol level
> changes (such as SRS) would do much, much more than body checks to make SPF
> not just feasible, but very convenient. Even new ESMTP extensions are
> probably more likely to be widely adopted than specialized body checks.

I'd still like to do such body checks on my own machine, even if it
means I'll have to work around buggy MTA's. I'm probably not the only
one who wants this. (Assuredly not, since it's been discussed here.)

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-24 02:32:01 UTC
Permalink
On Fri, 23 Jan 2004, Mark Shewmaker wrote:

: The tests discussed elsewhere on this list in the past week or so
: discussing the possibility of neither header body Sender: nor From:
: having anything to do with the MAIL FROM sender.

They don't in today's world as it is, nor should they. From: indicates an
origin; Sender: optionally indicates an aggregator or gateway; the envelope
indicates an authoritative DSN return path.

The envelope is most interesting from a forgery (and spam) control
perspective because it is very easy upon which to clamp down. It has
[RFC-mandated, for that matter] traceability implications that are not
inherent to From: or Sender:.

: > I didn't mean that. What I said (explained here in further detail) is that
: > rejection at DATA requires consuming the whole message, whereas dropping the
: > TCP connection is equivalent to a transient failure that will cause the
: > remote MTA to retry.
:
: But my understanding was, especially given other emails on this list
: stating this in no certain terms, was that if an MTA considered that a
: transient failure that that was a bug in that MTA

A dropped TCP connection in the middle of DATA is explicitly defined by
RFC821:4.1.1 as equivalent to a 4xx error, not a 5xx error.

====>
If the connection is closed prematurely the receiver should
act as if a RSET command had been received (canceling any
pending transaction, but not undoing any previously
completed transaction), the sender should act as if the
command or transaction in progress had received a temporary
error (4xx).
<====

This is further clarified by RFC2821:3.9, with rules on server and client
behaviors.

====>
An SMTP server MUST NOT intentionally close the connection except:

- After receiving a QUIT command and responding with a 221 reply.

- After detecting the need to shut down the SMTP service and
returning a 421 response code. This response code can be issued
after the server receives any command or, if necessary,
asynchronously from command receipt (on the assumption that the
client will receive it after the next command is issued).

[...]

SMTP clients that experience a connection close, reset, or other
communications failure due to circumstances not under their control
(in violation of the intent of this specification but sometimes
unavoidable) SHOULD, to maintain the robustness of the mail system,
treat the mail transaction as if a 451 response had been received and
act accordingly.
<====

The behavior described above (which is explicit in RFC821 and a "SHOULD"
in RFC2821) is indeed implemented as described in all currently deployed
SMTP MTAs, to my knowledge.

: The greater expense would come not from larger body data streams but
: because on the very small percentage of emails that make it past your
: envelope tests but not the simple spf body tests: You'll have to
: workaround buggy MTA's by keeping track of these past failures to reject
: at envelope the second time around, or use some other expensive
: workaround.

If it makes it past an envelope check, then something major is amiss, and
the whole shebang is a prime candidate for education, scolding, and/or
blacklisting (so that it *does* get caught by many folks at envelope test
time). We already do this sort of thing in the wild today.

: > Adding a bunch of DATA phase check possibilities to the core spec encourages
: > too many mail senders (you know, the ones who create the SPF records in DNS)
: > to be lazy. This defeats the lightweight check purpose of SPF, which I
: > believe will lead to too many "unknown" results in the wild due to low
: > adoption rate.
:
: Honestly that doesn't make sense. I really don't follow the logic and I
: don't understand what you mean by being lazy: I can't imagine that
: anyone would ever want to skimp on the envelope checks for one, and the
: header checks discussed elsewhere are about as clear as the envelope
: ones.

Encouraging use of header/body checks, which is what putting them in the
main spec would do, defeats the purpose of having lightweight envelope-time
checks. The primary, encouraged method should be the one that lives up to
the [original] lightweight envelope check mission of SPF.

Body-based checks would tend to be easier (hence "lazy") to implement for
less knowledgeable folk, since the From: header is typically more prominent
than the envelope and seems to some as the authoritative place to go. If a
large number of people stuck only DATA-phase checks in their SPF records,
and -- as has been shown in past DATA-phase filtering efforts -- ISPs don't
have the horsepower or contractual ability to check at DATA time, you've
actually done them a disservice.

Those DATA-only SPF records would, instead, return an "unknown" result, thus
equivalent to having no SPF record at all. That puts one hell of a stick in
the mud wrt SPF's goal of being deterministic for the majority of domains.

: > Please, before we fall into the DATA-phase trap, put it in an extension
: > document that is not attached to the core SPF spec.
:
: Uhm, that's effectively the same thing I'm suggesting--I really can't
: see the difference between a separate extension and mention of how to do
: an additional check that people will surely be doing.
^^^^^^^^^^^^^^^^^^^^^^^^^^^

The majority of the anti-spam community balks at any kind of new DATA-phase
check, particularly if it's meant to be somehow authoritative.

If my word's not good enough, I wholeheartedly encourage you to ask them
yourself. But an advance word of warning: In this thread, I've been quite
... tactful ... compared to some others you may run into. 8-)

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-26 16:20:01 UTC
Permalink
On Fri, 2004-01-23 at 21:32, tv+***@duh.org wrote:
> On Fri, 23 Jan 2004, Mark Shewmaker wrote:
>
> the envelope
> indicates an authoritative DSN return path.
>
> The envelope is most interesting from a forgery (and spam) control
> perspective because it is very easy upon which to clamp down.

Error in my thinking: I thought MTAs threw MAIL FROM away.

I did not realize that it was preserved in Return-Path:

As the original and spf-validated MAIL FROM would be available in the
Return-Path: for MTA's or filtering programs to compare on as they like,
I am no longer concerned about the spf1 spec being unable to attain its
own goals, or the would-be-resultant need to soon do mta-level header
checks as spammers adapt, checks that the spec neither requires, nor
enourages, nor mentions.

My concerns on this point was entirely due to my own misunderstanding.

However, for things such as domain-keys checks and similar proposals, I
think that there will still be a general desire to have future spf specs
discuss how to do body checks, so I'll continue this response in that
light.

That is, a discussion of the technical merits or stupidities :-) of
rejects-at-"." for when people are clamoring for it and claim they want
it.

But first, I have to admit to yet another mistake:

> : > rejection at DATA requires consuming the whole message, whereas dropping the
> : > TCP connection is equivalent to a transient failure that will cause the
> : > remote MTA to retry.
> :
> : But my understanding was, especially given other emails on this list
> : stating this in no certain terms, was that if an MTA considered that a
> : transient failure that that was a bug in that MTA
>
> A dropped TCP connection in the middle of DATA is explicitly defined by
> RFC821:4.1.1 as equivalent to a 4xx error, not a 5xx error.

I misread: When you mentioned "dropping the TCP connection" above I
simply read right through that without comprehending what you actually
said, somehow reading instead that you were again referring to the same
sort of rejection-at-"." that I and everyone has been talking about,
namely, returning a 5xx error.

I simply didn't catch that you were referring to actually dropping the
TCP connection itself!

I am most emphatically not suggesting that! I was instead suggesting
returning a 5xx error at that point. 571 sounds spot-on to me.

And returning errors after DATA is okay according to RFC2821, section
3.3:

| The end of mail data indicator also confirms the mail transaction and
| tells the SMTP server to now process the stored recipients and mail
| data. If accepted, the SMTP server returns a 250 OK reply. The DATA
| command can fail at only two points in the protocol exchange:
|
| - [...]
|
| - If the verb is initially accepted and the 354 reply issued, the
| DATA command should fail only if the mail transaction was
| incomplete (for example, no recipients), or if resources were
| unavailable (including, of course, the server unexpectedly
| becoming unavailable), or if the server determines that the
| message should be rejected for policy or other reasons.

Of course if MTA's don't respond properly to 5XX codes after DATA, (by
considering them transient failures instead of permanent), then that
problem would have to be worked around somehow, (and I suggested one way
to do so.)

Because I did misread you a couple messages ago, I think it makes sense
to back up and more appropriately re-respond to your previous message:

> : > What I said (explained here in further
> : > detail) is that rejection at DATA requires consuming the whole
> : > message, whereas dropping the TCP connection is equivalent to a
> : > transient failure that will cause the remote MTA to retry. So any
> : > check that is at DATA is inherently more expensive by several
> : > orders of magnitude, as the entire data stream must be received
> : > before the 5xx can be sent.

Two things:

1. The first sentence compares the rejection of the forgery (which
as discussed in other messages this week would be done via a 5xx
error code) with dropping the TCP connection.

Where does dropping the TCP connection ever come into the picture?

(I'm guessing you also misunderstood me. If that's so we can drop
the whole dropped-connections confusion.)

2. You then claim that a check at DATA is more expensive, as the
entire data stream must be sent.

My question is "More expensive than what?":

o More expensive than an envelope check?

Nope: We've already done an envelope check, the message so far
passed it, and so we're now obliged to receive the entire data
stream anyway.

Doing a check after we get the data we're obliged to get isn't
any more bandwidth expensive than NOT doing a check after we get
the data we're obliged to get.

o Dropping the TCP connection? No, because even if you wanted
to do that for some reason, you'd just get the same email again
later, so dropping the TCP connection would be more bandwidth
expensive.

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-26 16:53:21 UTC
Permalink
On Mon, 26 Jan 2004, Mark Shewmaker wrote:

: I misread: When you mentioned "dropping the TCP connection" above I
: simply read right through that without comprehending what you actually
: said, somehow reading instead that you were again referring to the same
: sort of rejection-at-"." that I and everyone has been talking about,
: namely, returning a 5xx error.

OK, clarification made. It's likely that I misinterpreted an earlier post
that referred to "dropping" a connection in relation to consuming all the
message at DATA time.

: 2. You then claim that a check at DATA is more expensive, as the
: entire data stream must be sent.
:
: My question is "More expensive than what?":
:
: o More expensive than an envelope check?
:
: Nope: We've already done an envelope check, the message so far
: passed it, and so we're now obliged to receive the entire data
: stream anyway.

Yes, but the expense comes under the assumption that a body check is perhaps
the *only* SPF method provided by the potentially forgeable domain in their
TXT record. If that's the case, then there is no envelope check at all, and
it is indeed more expensive to process that message's SPF authenticity.

This is where the whole concept about "allowing senders to be lazy" comes
from. If SPF-authenticated senders are permitted (or worse yet, encouraged)
to do body-only checks, then the lightweight value of SPF is dead in the
water. If using SPF, envelope checks should be mandatory, and body checks
optional as a [secondary] method of reassurance only.

(And, once you reach that point, why include body checks in SPF at all?
What would it buy you?)

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-26 17:37:38 UTC
Permalink
On Mon, 2004-01-26 at 11:53, tv+***@duh.org wrote:
> On Mon, 26 Jan 2004, Mark Shewmaker wrote:
>
> : 2. You then claim that a check at DATA is more expensive, as the
> : entire data stream must be sent.
> :
> : My question is "More expensive than what?":
> :
> : o More expensive than an envelope check?
> :
> : Nope: We've already done an envelope check, the message so far
> : passed it, and so we're now obliged to receive the entire data
> : stream anyway.
>
> Yes, but the expense comes under the assumption that a body check is perhaps
> the *only* SPF method provided by the potentially forgeable domain in their
> TXT record. If that's the case, then there is no envelope check at all, and
> it is indeed more expensive to process that message's SPF authenticity.

I can agree with that.

I was not suggesting that there be any changes in spf1 TXT records. I
was not suggesting that spf1 text records have any reference to
body-versus-header--merely that the standard require and document
envelope checks, and allow and document header/body checks.

That being the case, the situation you mention could not occur for spf1.

However, to be fair, a big reason to allow for body checks in future spf
versions would be to allow for that sort of thing. I was and am hoping
for some spf revision to allow for something such as:

1. "Consider mail purporting to be from my domain to be from my domain
if sent from any of these IPs."
2. "Also consider mail purporting to be from my domain and not from
the above IP's, but containing mumble-mumble signature check in the
headers, to be from my domain."
3. "Otherwise, do not believe any emails that claim to be from my
domain."

So your point is spot-on in that way, as such a method would allow for
lazy domains to only publish public-key or fingerprint type data and no
simple envelope-check type data. It would be...annoying if domains did
that. (As well as an odd sort of lazyness to set up public key
signature fingerprints and valid times and what have you, but not
actually just type in a few valid sending IPs.)

I would like future spf proposals to allow for that sort of traveling
salesman scenario--such sending emails from a domain even if the
authorized outgoing mailservers are down. Intricate debate on how to do
that is probably not timely now, given spf1 spec cleanup. (Although I'm
willing to participate of course. :-) )

However, I would hope that it would be possible to do something like the
above, without the lazy-domain problem you're worried about. (Possibly
the spec disallowing listing keys without also listing valid outgoing
domains; I don't know.)

There has to be some way around the problem you legitimately worry
about--perhaps spf will help to limit the spam problem to such an
extent, freeing up resources on mail servers, that doing mta-level body
checks on every single email will no longer be considered too much
work. :-)

Anyway, your worry not valid for what I was actually proposing at the
time, but it is valid for what I hope can be done at some point.

> This is where the whole concept about "allowing senders to be lazy" comes
> from. If SPF-authenticated senders are permitted (or worse yet, encouraged)
> to do body-only checks, then the lightweight value of SPF is dead in the
> water. If using SPF, envelope checks should be mandatory, and body checks
> optional as a [secondary] method of reassurance only.
>
> (And, once you reach that point, why include body checks in SPF at all?
> What would it buy you?)

I agree for spf1 type checks, especially with my corrected Return-Path:
misunderstanding. (And thank you for continuing to clarify your
reasoning in the face of that hidden misunderstanding.)

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-26 18:28:33 UTC
Permalink
On Mon, 26 Jan 2004, Mark Shewmaker wrote:

: > Yes, but the expense comes under the assumption that a body check is perhaps
: > the *only* SPF method provided by the potentially forgeable domain in their
: > TXT record. If that's the case, then there is no envelope check at all, and
: > it is indeed more expensive to process that message's SPF authenticity.
:
: I can agree with that.

: However, to be fair, a big reason to allow for body checks in future spf
: versions would be to allow for that sort of thing.

And why is that...?

: 2. "Also consider mail purporting to be from my domain and not from
: the above IP's, but containing mumble-mumble signature check in the
: headers, to be from my domain."

Please, for the love of Jon Postel, NO.

(And policy issues aside, I already know that a "signature check" sender
verification flag is likely to get the boot by the majority of admins, thus
reducing said check to the short circuit "unknown" result. Ick.)

: I would like future spf proposals to allow for that sort of traveling
: salesman scenario

There has been a correct fix for the "traveling salesman" problem for the
better part of a decade now. It's called "authenticated SMTP," which can
refer to SASL or the earlier POP/IMAP-before-SMTP. Large ISPs have
implemented this correct fix now for some time. I would really rather *not*
see backpeddling on this front, as it has made SMTP more reliable and
traceable by several orders of magnitude.

Do you have some other example where body checks would be useful? I'm at a
loss to understand why anyone should be publishing such records.

What we want is as few "unknown" results as possible, and with what I've
observed of admins wrt message-body checks, adding such things to a later
revision of SPF is likely to increase "unknown"s in the long run.

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark Shewmaker
2004-01-29 18:41:49 UTC
Permalink
On Mon, 2004-01-26 at 13:28, tv+***@duh.org wrote:
> On Mon, 26 Jan 2004, Mark Shewmaker wrote:
>
> : I would like future spf proposals to allow for that sort of traveling
> : salesman scenario

You deleted: "such [as] sending emails from a domain even if the
authorized outgoing mailservers are down."

> There has been a correct fix for the "traveling salesman" problem for the
> better part of a decade now. It's called "authenticated SMTP," which can
> refer to SASL or the earlier POP/IMAP-before-SMTP.

--
Mark Shewmaker
***@primefactor.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-29 20:02:07 UTC
Permalink
On Thu, Jan 29, 2004 at 01:41:49PM -0500, Mark Shewmaker wrote:
| On Mon, 2004-01-26 at 13:28, tv+***@duh.org wrote:
| > On Mon, 26 Jan 2004, Mark Shewmaker wrote:
| >
| > : I would like future spf proposals to allow for that sort of traveling
| > : salesman scenario
|
| You deleted: "such [as] sending emails from a domain even if the
| authorized outgoing mailservers are down."

Philip Gladstone posted a rate limiting thing so each user could send N
messages from a given IP per day.

You can also do a per-user exemption using so some users are allowed to
send mail from any IP.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
t***@duh.org
2004-01-29 20:32:52 UTC
Permalink
On Thu, 29 Jan 2004, Mark Shewmaker wrote:

: > : I would like future spf proposals to allow for that sort of traveling
: > : salesman scenario
:
: You deleted: "such [as] sending emails from a domain even if the
: authorized outgoing mailservers are down."

Not something you want to allow opening on a wide basis. You can do
per-user macros, as well as provide specific alternates--say, the dialup ISP
with which said organization purchases accounts for use. That still doesn't
need header/body checks.

SPF is meant to provide a good SMTP transaction level framework. Let's try
to work inside that and come up with better ways of handling corner cases so
that the likelihood of "unknown" coming from an unimplemented mechanism in
the wild stays low.

--
-- Todd Vierling <***@duh.org> <***@pobox.com>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Dan Willett Sr.
2004-01-26 17:54:15 UTC
Permalink
Ok, all of this is interesting, and I personally think additional headers are
a better idea than re-writing headers on forwarding...

But what I want to know, is there a step by step faq to impliment SPF on
Sendmail for my FreeBSD run mailserver?

I've downloaded the SendMail milter... but it does not have any instructions
on how to install it. And no, I don't want to have to spend time learning
all the ins and outs of Sendmail... Personally, I don't like sendmail, and I
don't use it on my personal machines, but right now it is what is on the mail
server that I manage. I've looked, but I haven't found any step-by-step
directions to install SPF on sendmail (or Exim or ... for that matter)

I believe there are probably a lot of people like me, who would impliment SPF
immediatly, but they need step-by-step directions. Not directions that
assume you know anything.

If these directions exist, please point me in their direction, and put a link
to them on the download page.

Thanks.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark
2004-01-26 18:32:25 UTC
Permalink
Hello Dan,

Yesterday I sent the INSTALL document to the devel list, for review. Meng
has not "officially" signed off on it yet, but there no reason I cannot
already distribute it here ("spf-milter-INSTALL.txt" as attachment). The
official document will either be this, or something very like it. :)

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx


----- Original Message -----
From: "Dan Willett Sr." <***@Willett.net>
To: <spf-***@v2.listbox.com>
Sent: Monday, January 26, 2004 7:05 PM
Subject: [spf-discuss] Implimenting SPF in Sendmail

> Ok, all of this is interesting, and I personally think additional headers
> are a better idea than re-writing headers on forwarding...
>
> But what I want to know, is there a step by step faq to impliment SPF on
> Sendmail for my FreeBSD run mailserver?
>
> I've downloaded the SendMail milter... but it does not have any
> instructions on how to install it. And no, I don't want to have to spend
> time learning all the ins and outs of Sendmail... Personally, I don't like
> sendmail, and I don't use it on my personal machines, but right now it is
> what is on the mail server that I manage. I've looked, but I haven't found
> any step-by-step directions to install SPF on sendmail (or Exim or... for
> that matter)
>
> I believe there are probably a lot of people like me, who would impliment
> SPF immediatly, but they need step-by-step directions. Not directions that
> assume you know anything.
>
> If these directions exist, please point me in their direction, and put a
> link to them on the download page.
>
> Thanks.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-26 18:38:05 UTC
Permalink
On Mon, Jan 26, 2004 at 06:32:25PM +0000, Mark wrote:
|
| Yesterday I sent the INSTALL document to the devel list, for review. Meng
| has not "officially" signed off on it yet, but there no reason I cannot
| already distribute it here ("spf-milter-INSTALL.txt" as attachment). The
| official document will either be this, or something very like it. :)
|
|
| If everything went okay, try 'ps ax', and your spf-milter will show up as:
|
| "spf-milter (perl)"
|
| Restart sendmail. Now you're done. :)
|

Great job Mark, it looks very comprehensive. I would run through the
procedure right now but unfortunately all my boxes are postfix, so I
can't actually test it.

It might be a good idea, though, to suggest at the very end a couple
ways to confirm that

1) legitimate mail gets through
2) forged mail does not (forging mail from ***@vw.mailzone.com will
do the trick)

It would help if we had auto probe engine that tests if a server's SPF
detection is working correctly so people don't have to pretend, by hand,
to be an SMTP client...

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
wayne
2004-01-26 18:43:18 UTC
Permalink
In <***@Willett.net> "Dan Willett Sr." <***@Willett.net> writes:

> I believe there are probably a lot of people like me, who would impliment SPF
> immediatly, but they need step-by-step directions. Not directions that
> assume you know anything.

A word of warning:

SPF implementations are currently still being developed, tested and
debugged. Almost all of them have been updated in the last few weeks,
and the ones that haven't are probably broken in some way.

My gut feel is that if you want/need step-by-step directions, you
probably shouldn't deploy SPF right now. Wait a few weeks and see how
things are going.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Greg Connor
2004-01-26 19:56:32 UTC
Permalink
--Mark Shewmaker <***@primefactor.com> wrote:
>
> Error in my thinking: I thought MTAs threw MAIL FROM away.
>
> I did not realize that it was preserved in Return-Path:


Interestingly, my sendmail/pop server shows "Return-Path:" (always at the
very top) but my Exchange server doesn't show Return-Path at all. Perhaps
Return-Path: is not consistent. Sort of bad news for SPF if it isn't.


--
Greg Connor <***@nekodojo.org>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
wayne
2004-01-26 20:01:26 UTC
Permalink
In <855023508.1075118192@[172.23.65.45]> Greg Connor <***@nekodojo.org> writes:

> Interestingly, my sendmail/pop server shows "Return-Path:" (always at
> the very top) but my Exchange server doesn't show Return-Path at all.
> Perhaps Return-Path: is not consistent. Sort of bad news for SPF if
> it isn't.


Adding the Return-Path: header is not required and many MTAs don't do
it. Some use Sender: instead, some many use things like
Envelope-From:. I forget all the variations.

A good place to find the answer would be to check out the development
version of SpamAssassin and/or ask Justin Mason.


However, if the MTA does SPF checks, it should generate a
Received-SPF: header which will include the envelope-from.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Mark
2004-01-26 20:40:00 UTC
Permalink
----- Original Message -----
From: "wayne" <***@midwestcs.com>
To: "SPF discussions" <spf-***@v2.listbox.com>
Sent: Monday, January 26, 2004 9:01 PM
Subject: Re: [spf-discuss] Is Return-Path as available as we think?


> However, if the MTA does SPF checks, it should generate
> a Received-SPF: header which will include the envelope-from.

That is an excellent point. Regardless of whether the native MTA exports the
envelope-from to the headers, the Received-SPF: header will ALWAYS contain
the envelope-from.

This whole discussion, btw, is not inherently different from, say, how to
extract the envelope-from when running procmail from a .forward file. And,
just like procmail, I believe SPF is therefore best run at MTA level; doing
a post-mortem extraction on headers to gain relevant information from the
SMTP exchange, such as envelope-from and recipients, has always been
questionable (and, in the case of the latter, downright untrustworthy). This
is a problem -- if you want to call it that -- intrinsic to the SMTP
protocol itself, as headers are part of the DATA phase, and may, or may not,
contain everything you want, or contain it in undistorted form.

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
j***@cleburne.com
2004-01-23 17:08:31 UTC
Permalink
<snip>

> I sure hope not. I might as well pack up on the SPF idea and go home if it
> turns into just another "filtering" scheme, because it'll end up being no
> more useful than all the other filtering I have to do today.
>
> SPF needs to provide determinism at the protocol level -- before a whole
> message is accepted.

I am in full agreement with this. And I am a mini-isp. :)

John Chapman

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Jeremy Doupe
2004-01-23 22:22:13 UTC
Permalink
> >
> > SPF needs to provide determinism at the protocol level -- before a
whole
> > message is accepted.
>
> I am in full agreement with this. And I am a mini-isp. :)
>

As an ISP representative, I would agree that SPF filtering should take
place prior to DATA, however, I don't think the spec should mandate
where filtering is to be done. That should be left up to the
implementer (since that is probably going to happen anyway).

-=Jeremy


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Aredridel
2004-01-22 02:30:00 UTC
Permalink
On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
> On Wed, Jan 21, 2004 at 05:53:46PM +0100, Alex van den Bogaerdt wrote:
> | >
> | > to be precise, it doesn't require the MTA to accept it. it requires the
> | > MTA to transfer it, which is a bandwidth cost. in theory the MTA can
> | > still reject it at "." time.
> |
> | Indeed. One of these days I'm going to learn how to write what I think.
> |
>
> Could ISPs please weigh in on whether your email/spam bandwidth cost is
> signicant or not, thanks.

Yes.

3/4 of the mail we handle is spam.

I'd save at least 1% of my total budget if spam dissapeared, and nearly
20% of my bandwidth budget.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
RK Mail LIst
2004-01-22 03:58:38 UTC
Permalink
we are getting 93% spam. Therefore, a week we handle over 210,000 messages
which are pure junk mail. We eliminated most of it by offloading it to
SpamSlayer's Network. Now we have saved about 200kbits of steady stream to
be used for co-location.

Rudy K.

----- Original Message -----
From: "Aredridel" <***@theinternetco.net>
To: <spf-***@v2.listbox.com>
Sent: Wednesday, January 21, 2004 9:30 PM
Subject: Re: [spf-discuss] Calling ISPs: does bandwidth matter to your
bottom line?


> On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
> > On Wed, Jan 21, 2004 at 05:53:46PM +0100, Alex van den Bogaerdt wrote:
> > | >
> > | > to be precise, it doesn't require the MTA to accept it. it requires
the
> > | > MTA to transfer it, which is a bandwidth cost. in theory the MTA
can
> > | > still reject it at "." time.
> > |
> > | Indeed. One of these days I'm going to learn how to write what I
think.
> > |
> >
> > Could ISPs please weigh in on whether your email/spam bandwidth cost is
> > signicant or not, thanks.
>
> Yes.
>
> 3/4 of the mail we handle is spam.
>
> I'd save at least 1% of my total budget if spam dissapeared, and nearly
> 20% of my bandwidth budget.
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your
subscription,
> please go to
http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
>

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Seth Goodman
2004-01-21 18:23:57 UTC
Permalink
Here's a couple of newbie questions that I couldn't find the answers to in
either your draft RFC or the FAQ, though I'm sure it's in there somewhere.

First question:

Does the SPF record go in the DNS of the domain registrar or the domain
hosting service? For example, my domain is registered at GoDaddy.com and
hosted at Interland. As one of the .com TLD registrars, GoDaddy points to
Interland's nameservers as authoritative for my domain. Where does the SPF
record belong?

Interland has so far refused to even consider SPF, has refused to tell me
the CIDR range that their few dozen outgoing MX's share (besides their
entire /14) and has also refused to create any TXT records to support SPF.
I assume that they will eventually come around, so I would like to come up
with a suitable SPF record.

Hence, my second question:

Though I don't have a reasonable CIDR range for their outgoing MX's, there
is a consistent naming structure that I could take advantage of with the PTR
mechanism. Their outgoing MX's are always named like:

mail[xx].atl.registeredsite.com

where the [xx] represents one or more decimal digits, for example:

mail4.atl.registeredsite.com


Rather than just using:

v=spf1 ptr:registeredsite.com -all

which is a whole /14, or the slightly better:

v=spf1 ptr:atl.registeredsite.com -all

I would like to use a macro in an SPF record to describe the MX names more
precisely. My idea is to use the *DIGIT macro feature, perhaps something
like

v=spf1 ptr:mail%{1}.atl.registeredsite.com -all

However, the draft RFC does not give any examples of expanding a macro
including *DIGIT, though that function is described. Would this SPF record
work with the MX name structure I gave? I can see that *DIGIT was meant for
extracting the parts of a dotted quad, but I hope it is applicable to this
problem, as well. I think an example in the draft RFC of how *DIGIT expands
might be helpful, particularly how it puts the separator back in or doesn't
and if it can be used for something other than a dotted quad.

My local ISP, whom I send some outgoing mail through, also uses a similar
naming scheme for their outgoing MX's, so I suspect that a solution to this
would be useful to others.

--
Seth Goodman

off-list replies to sethg [at] GoodmanAssociates [dot] com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Kelson Vibber
2004-01-21 18:56:05 UTC
Permalink
At 10:23 AM 1/21/2004, Seth Goodman wrote:
>For example, my domain is registered at GoDaddy.com and hosted at
>Interland. As one of the .com TLD registrars, GoDaddy points to
>Interland's nameservers as authoritative for my domain. Where does the
>SPF record belong?

Interland. Think of it the same way you'd ask to point www.yourdomain.com
to a new IP address, or define a new MX record.

>Interland has so far refused to even consider SPF, has refused to tell me
>the CIDR range that their few dozen outgoing MX's share (besides their
>entire /14) and has also refused to create any TXT records to support SPF.

Wait, you're paying them for DNS hosting, but they refuse to create DNS
records for you? It should be fairly simple: "I want the following TXT
record for mydomain.com: v=spf1 blah blah blah" and they should create it,
whether they "support" SPF or not.

If they don't want to add records for interland.com, that's their business,
but they should be willing to add any DNS record you want for
yourdomain.com if they host it for you.

Incidentally, you only need to know Interland's IP addresses if you use
them to send outgoing mail. If you send mail through abc.net, and use
xyz.com to host your DNS record, you only need to know abc.net's outgoing
mail servers. Then you build an SPF record with that info, and ask xyz.com
to add it as a TXT record for yourdomain.com.


Kelson Vibber
SpeedGate Communications <www.speed.net>


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Seth Goodman
2004-01-21 19:08:15 UTC
Permalink
This post might be inappropriate. Click to display it.
wayne
2004-01-22 05:49:32 UTC
Permalink
In <***@spamcop.net> "Seth Goodman" <***@spamcop.net> writes:

> Interland's outgoing MX's are distinct from their web hosting clusters. All
> their MX's, incoming, outgoing and transfer, operate from the
> registeredsite.com/14 netblock, not from anything related to the
> Interland.com IP addresses. A /14 netblock is so big that it is at best a
> weak test for forgeries.

Several things in semi-random order.

First off, a /14 isn't really that large. More importantly, if any
Interland.com customer starts forging your domain name, I would hope
that Interland.com would be willing to do something about it.


That said, it appears that Interland.com has more than a /14. Doing a
"dig interland.com" shows that their name servers are on 64.226.28.33,
64.77.127.42, and 69.0.145.33, which is quite a spread.

The best tool that I know of to figure this stuff is to use
senderbase.org. If anyone knows a better tool, please let me know!

For example, a search on interland.com shows quite a few IP addresses
that send email. See:
http://www.senderbase.org/search?searchString=interland.com

This shows only one IP address that is sending email within the
interland.com domain name. However, it does list the network operator
of INTERLAND. Clicking on that link or typing that name into the
search box yeilds:
http://www.senderbase.org/search?searchBy=organization&searchString=INTERLAND

>From there, it appears that you are correct, most of the email is
coming from the domain name of registerdsite.com. More over, it looks
like you are pretty safe with using ptr:atl.registeredsite.com and/or
ip4:64.224.219.0/24



Anyway, this is how I've been tracking down this kind of thing. If
anyone has other techniques for figuring out random domain's *OUTPUT*
SMTP servers, please let me know!


-wayne



-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Seth Goodman
2004-01-22 17:21:44 UTC
Permalink
Thanks for the very useful info, both Wayne and John.

> [John Warren]
> So forget their DNS servers and go to zoneedit.com and do your own.
> It's easy, the only thing you can't do is "_SPF" since they don't allow
> the "_".

That's a reasonable option. What is the "_SPF" used for? I was just going
to use the ptr mechanism to qualify my outgoing MX's.


> [wayne]
> First off, a /14 isn't really that large. More importantly, if any
> Interland.com customer starts forging your domain name, I would hope
> that Interland.com would be willing to do something about it.

True on both counts. I often forget the distinction between what would be
"nice" and what is a reasonable practical solution.


> [wayne]
> That said, it appears that Interland.com has more than a /14. Doing a
> "dig interland.com" shows that their name servers are on 64.226.28.33,
> 64.77.127.42, and 69.0.145.33, which is quite a spread.

That's true, but for my class of puny hosting account (fixed IP, shared
server), all mail goes through their registeredsite.com mail cluster.


> [wayne]
> The best tool that I know of to figure this stuff is to use
> senderbase.org. If anyone knows a better tool, please let me know!

That's an amazing tool, thanks. I was amused to see their individual
outgoing MX's listed on the web, when after several escalations of my
request for this information, a senior tech support person for email
categorically stated that they would never give out the IP's or the CIDR
range of their outgoing MX's due to security considerations.

I tried to diplomatically sell the benefits of SPF during the process of
requesting either a CIDR range or a naming structure for their outgoing
MX's, but was finally told that they looked at SPF, decided that it had no
industry support and did not plan to make provisions for it. From the
sidelines, that does not appear to be true, and I hope that they reconsider
SPF or whatever verification scheme the industry eventually gets behind. As
a large hosting service, they tend to be late adopters. For the past year,
they have been filtering or tagging email, configurable per domain, against
both SpamCop and Sorbs (not configurable). They are just now considering
rejection at the SMTP level against those DNSBL's, but again, all or
nothing. They have thus far decided against any kind of secure
authentication or support for port 587. I still think they'll come around,
but not quickly.


> [wayne]
> From there, it appears that you are correct, most of the email is
> coming from the domain name of registeredsite.com. More over, it looks
> like you are pretty safe with using ptr:atl.registeredsite.com and/or
> ip4:64.224.219.0/24

That was the /24 I was thinking of based on my own observations. The
senderbase tool verifies this. I agree that either of your solutions is
fine for my purposes.

--
Seth Goodman

off-list replies to sethg [at] GoodmanAssociates [dot] com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Seth Goodman
2004-01-21 19:25:47 UTC
Permalink
> At 10:23 AM 1/21/2004, Seth Goodman wrote:
> >For example, my domain is registered at GoDaddy.com and hosted at
> >Interland. As one of the .com TLD registrars, GoDaddy points to
> >Interland's nameservers as authoritative for my domain. Where does the
> >SPF record belong?
>
> [Kelson Vibber]
> Interland. Think of it the same way you'd ask to point
> www.yourdomain.com
> to a new IP address, or define a new MX record.

I forgot to thank you for answering this question, so thanks!

--
Seth Goodman

off-list replies to sethg [at] GoodmanAssociates [dot] com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
John Warren
2004-01-22 07:29:16 UTC
Permalink
So forget their DNS servers and go to zoneedit.com and do your own.
It's easy, the only thing you can't do is "_SPF" since they don't allow
the "_".


On 21 Jan 2004 at 13:08, Seth Goodman wrote:

> > [Kelson Vibber]
> > Wait, you're paying them for DNS hosting, but they refuse to create DNS
> > records for you? It should be fairly simple: "I want the following TXT
> > record for mydomain.com: v=spf1 blah blah blah" and they should
> > create it,
> > whether they "support" SPF or not.
> >
> > If they don't want to add records for interland.com, that's their
> > business,
> > but they should be willing to add any DNS record you want for
> > yourdomain.com if they host it for you.
>
> They _should_ be willing, but they are not. They support user control of A,
> MX and CNAME records and that's it. They are a large hosting service, and
> their attitude is pretty much, "My way or the highway".
>
> > [Kelson Vibber]
> > Incidentally, you only need to know Interland's IP addresses if you use
> > them to send outgoing mail. If you send mail through abc.net, and use
> > xyz.com to host your DNS record, you only need to know abc.net's outgoing
> > mail servers. Then you build an SPF record with that info, and
> > ask xyz.com
> > to add it as a TXT record for yourdomain.com.
>
> Interland's outgoing MX's are distinct from their web hosting clusters. All
> their MX's, incoming, outgoing and transfer, operate from the
> registeredsite.com/14 netblock, not from anything related to the
> Interland.com IP addresses. A /14 netblock is so big that it is at best a
> weak test for forgeries. That is why I was looking for a way to narrow down
> the range without knowing the actual CIDR address range. From experience, I
> can take an educated guess as the /24 they use for outgoing MX's, but they
> won't validate it so I can't depend on it.
>
> --
> Seth Goodman
>
> off-list replies to sethg [at] GoodmanAssociates [dot] com
>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com


----------------------------------------------------------------------
John Warren, President | Tel.: +1 714-573-9650
Warren Engineering | Fax: +1 714-573-9289
14681 Danborough Rd. | mailto:***@wenet.tustin.ca.us
Tustin, CA 92780-6755 | mailto:***@wenet.tustin.ca.us
| http:Someday.com
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
arnold+spf-QMRIvgJGioDQT0dZR+
2004-01-20 22:44:14 UTC
Permalink
> On Tue, Jan 20, 2004 at 11:45:00AM +0100, Julian Mehnle wrote:
>
> > I don't quite understand the "processing headers is costly" argument
>
> In order to be able to process the headers, they have to be received
> in the first place.
>
> The difference is that in one kind of proposal one is able to stop
> a mail right after one of the RFC 821 type of commands (upto and
> including "DAtA") whereas in the other kind of proposal one has to
> receive the attached message (which could be huge!), process it and
> delete it.
>
> The difference is not in processing itself (at least: no significant
> difference) but in transfering the body or not.
>
> HTH
> Alex

An even bigger deal than the bandwidth and processing is the
administration of non-delivery notifications. If I can reject the
message during the SMTP dialogue, then my MTA has not taken
responsibility for the message. The previous MTA in the change has to
send the NDN and the postmaster of *that* MTA gets a copy.

If my MTA receives the message (takes responsibility with a 2xx response
to the DATA command), then my MTA must send the NDN and *I* get a copy.
Yuch!

Regards,
"Steve" Stephen L. Arnold, Ph.D., President, Arnold Consulting, Inc.
Address 2530 Targhee Street, Madison, Wisconsin 53711-5491 U.S.A.
Telephone +1 608 278 7700 Facsimile +1 608 278 7701
Internet Stephen.L.Arnold-QMRIvgJGioDQT0dZR+***@public.gmane.org http://WWW.Arnold.com
Arnold® is a registered trademark and service mark of Arnold Consulting, Inc.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.orgbox.com
Alex van den Bogaerdt
2004-01-20 22:56:00 UTC
Permalink
On Tue, Jan 20, 2004 at 04:44:14PM -0600, arnold+***@Arnold.com wrote:

> If my MTA receives the message (takes responsibility with a 2xx response
> to the DATA command), then my MTA must send the NDN and *I* get a copy.
> Yuch!

Another kind of cost, but yes: This is also true. Even worse again,
your MTA will probably be stuck with the DSN as it will probably be
undeliverable.

cheers,
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
John Warren
2004-01-20 13:01:49 UTC
Permalink
On 20 Jan 2004 at 11:45, Julian Mehnle wrote:

> Meng Weng Wong [mengwong-GXA0fb4l1L89V/L/***@public.gmane.org] wrote:
> > On Mon, Jan 19, 2004 at 09:25:11PM -0500, Marc Alaia wrote:
> > > I think that an unreasonably-high priority is being put on
> > > terminating the message before full receipt. The cost-factor may be
> > > high now, but once SPF is more prevalent spammers will not send
> > > non-SPF-compliant emails knowing that they will be simply deleted by
> > > the receiving system and therefore doing it this way will eventually
> > > be worthless. I'd rather we do it right in the beginning than having
> > > less perfect system because one group of users sees it more
> > > economically right now....
> >
> > This could be true. What do people think will happen? Any spammers on
> > the list please speak up, your input is important. :)
> >
> > But processing headers is costly. And I don't know if worms and
> > viruses will care.
>
> "Once SPF is more prevalent", who will write virm that forge the
> sender address, knowing that the messages will be rejected/deleted
> automatically(!) anyway?
>
> I don't quite understand the "processing headers is costly" argument,
> though. Why would SPF need to look at the headers in the first place?
> If I understand John Warren's proposal correctly, it essentially just
> says that forwarding/bounce information should be put into the
> "Sender:" header field instead of using a complicated rewriting scheme
> for the envelope sender. The core functionality of SPF, i.e.
> determining if the envelope sender domain is "valid", isn't touched by
> that change, is it?

Actually the "MAIL FROM" envlope header and copied to the "Sender". I
did make aN error here though. The "Return-path" header already should
have a copy of the "MAIL FROM".

>
> -------
> Sender Permitted From: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org


----------------------------------------------------------------------
John Warren, President | Tel.: +1 714-573-9650
Warren Engineering | Fax: +1 714-573-9289
14681 Danborough Rd. | mailto:jwarren-***@public.gmane.org
Tustin, CA 92780-6755 | mailto:info-***@public.gmane.org
| http:Someday.com
+--------------------------------------------------------------------+
| Any and all use of my email address for bulk email without my |
| expressed permission is prohibited. This means NO JUNK EMAIL, SPAM.|
| Support the anti-Spam amendment, Join at http://www.cauce.org/ |
+--------------------------------------------------------------------+

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-discuss-***@public.gmane.org
Marc Alaia
2004-01-20 23:22:51 UTC
Permalink
I disagree. Treat it as spam and delete. My spam utility doesn't send
NDA's. The email was delivered, then deleted! My opinion is that this
belongs more in the post-delivery processing, such as spam utilities, than
in the mail-delivery process....

Marc

-----Original Message-----
From: arnold+***@Arnold.com [mailto:arnold+***@Arnold.com]
Sent: Tuesday, January 20, 2004 5:44 PM
To: spf-***@v2.listbox.com
Cc: arnold+***@Arnold.com
Subject: Re: [spf-discuss] rejection at SMTP time?


> On Tue, Jan 20, 2004 at 11:45:00AM +0100, Julian Mehnle wrote:
>
> > I don't quite understand the "processing headers is costly" argument
>
> In order to be able to process the headers, they have to be received
> in the first place.
>
> The difference is that in one kind of proposal one is able to stop
> a mail right after one of the RFC 821 type of commands (upto and
> including "DAtA") whereas in the other kind of proposal one has to
> receive the attached message (which could be huge!), process it and
> delete it.
>
> The difference is not in processing itself (at least: no significant
> difference) but in transfering the body or not.
>
> HTH
> Alex

An even bigger deal than the bandwidth and processing is the
administration of non-delivery notifications. If I can reject the
message during the SMTP dialogue, then my MTA has not taken
responsibility for the message. The previous MTA in the change has to
send the NDN and the postmaster of *that* MTA gets a copy.

If my MTA receives the message (takes responsibility with a 2xx response
to the DATA command), then my MTA must send the NDN and *I* get a copy.
Yuch!

Regards,
"Steve" Stephen L. Arnold, Ph.D., President, Arnold Consulting, Inc.
Address 2530 Targhee Street, Madison, Wisconsin 53711-5491 U.S.A.
Telephone +1 608 278 7700 Facsimile +1 608 278 7701
Internet ***@Arnold.com http://WWW.Arnold.com
Arnold® is a registered trademark and service mark of Arnold Consulting,
Inc.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to
http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-21 00:00:13 UTC
Permalink
On Tue, Jan 20, 2004 at 06:22:51PM -0500, Marc Alaia wrote:
| I disagree. Treat it as spam and delete. My spam utility doesn't send
| NDA's. The email was delivered, then deleted! My opinion is that this
| belongs more in the post-delivery processing, such as spam utilities, than
| in the mail-delivery process....

I think it's important to be FP friendly.

If rejection happens at SMTP time the legitimate but misconfigured
sender gets a bounce with instructions.

Note that you can still send a rejection after the "." so in theory you
can still parse the message but reject. In practice this seems to be
rather difficult. Most MTAs aren't architected to make that easy.


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
George Schlossnagle
2004-01-21 01:20:44 UTC
Permalink
On Jan 20, 2004, at 7:00 PM, Meng Weng Wong wrote:

> On Tue, Jan 20, 2004 at 06:22:51PM -0500, Marc Alaia wrote:
> | I disagree. Treat it as spam and delete. My spam utility doesn't
> send
> | NDA's. The email was delivered, then deleted! My opinion is that
> this
> | belongs more in the post-delivery processing, such as spam
> utilities, than
> | in the mail-delivery process....
>
> I think it's important to be FP friendly.
>
> If rejection happens at SMTP time the legitimate but misconfigured
> sender gets a bounce with instructions.

Doesn't this make it a vehicle for joe-jobs? Most service providers
are moving away from out-of-band DSNs for their expense and potential
for abuse. Or am I misunderstanding what you are saying?

George

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Meng Weng Wong
2004-01-21 01:34:45 UTC
Permalink
On Tue, Jan 20, 2004 at 08:20:44PM -0500, George Schlossnagle wrote:
| >I think it's important to be FP friendly.
| >
| >If rejection happens at SMTP time the legitimate but misconfigured
| >sender gets a bounce with instructions.
|
| Doesn't this make it a vehicle for joe-jobs? Most service providers
| are moving away from out-of-band DSNs for their expense and potential
| for abuse. Or am I misunderstanding what you are saying?
|

If the border MTA performs spoof detection, it can return a reject code
at SMTP time.

Bounce generation is the responsibility of the upstream server.
An SPF MTA is not required to generate a bounce.

If the upstream server is the original sender domain, that server will
generate a bounce.

If the upstream server is a spammer, it will not generate a bounce.

Either way, we are absolved of responsibility.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
George Schlossnagle
2004-01-21 01:38:28 UTC
Permalink
On Jan 20, 2004, at 8:34 PM, Meng Weng Wong wrote:

> On Tue, Jan 20, 2004 at 08:20:44PM -0500, George Schlossnagle wrote:
> | >I think it's important to be FP friendly.
> | >
> | >If rejection happens at SMTP time the legitimate but misconfigured
> | >sender gets a bounce with instructions.
> |
> | Doesn't this make it a vehicle for joe-jobs? Most service providers
> | are moving away from out-of-band DSNs for their expense and potential
> | for abuse. Or am I misunderstanding what you are saying?
> |
>
> If the border MTA performs spoof detection, it can return a reject code
> at SMTP time.
>
> Bounce generation is the responsibility of the upstream server.
> An SPF MTA is not required to generate a bounce.
>
> If the upstream server is the original sender domain, that server will
> generate a bounce.
>
> If the upstream server is a spammer, it will not generate a bounce.
>
> Either way, we are absolved of responsibility.

OK, we're on the same page.

George

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Kelson Vibber
2004-01-21 01:42:53 UTC
Permalink
At 05:20 PM 1/20/2004, you wrote:
>>If rejection happens at SMTP time the legitimate but misconfigured
>>sender gets a bounce with instructions.
>
>Doesn't this make it a vehicle for joe-jobs? Most service providers are
>moving away from out-of-band DSNs for their expense and potential for
>abuse. Or am I misunderstanding what you are saying?

No, when you reject at SMTP time, your server does not generate a bounce
message. All it does is issue a reject code and 1-line comment to the
connecting system, whether it's a mail client or another server.

If you end up with a DSN, it's generated somewhere further back in the chain.


Kelson Vibber
SpeedGate Communications <www.speed.net>


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Richard Kay
2004-01-21 23:18:08 UTC
Permalink
>On Wed, Jan 21, 2004 at 01:04:31PM -0500, Meng Weng Wong wrote:
>| Could ISPs please weigh in on whether your email/spam bandwidth cost is
>| signicant or not, thanks.

>The decision at hand is:
>How important is it to reject forgeries before DATA vs after DATA at "."
>time.
>If it is important, that's an argument in favour of the envelope sender,
>otherwise the argument allows use of headers.

From my point of view, the time cost involved in receiving a spam
I don't want that gets through my filters into my inbox, (between 3 and
30 seconds depending on whether I can reject it from the subject line
or have to open it first to manually see the content) is massive compared to
the trivial cost of the bandwidth involved in my MTA receiving the
whole thing. (I think I am well below my monthly fixed bandwidth
allocation. If so, the marginal bandwidth cost of receiving one extra
spam is, for me, zero.) If getting the data gives me a better perfomance
automated spam filter this must be worth the extra cost. I can understand
very large ISPs might see this differently, but how much is their customers'
lost time costing them in lost business ?

Another possible advantage of late automated spam detection is the
ability to set finer-grained policies for individual users at the
same domain or getting mail through the same virtual hosting MTA.

A further possible advantage is that it is possible to check
whether originating IPs have got into blacklists after the MTA
received the message, but when user logs in before opening their inbox.

Let's also look at this from the economics of the spamming operation.
If spammers have to try 10 envelope dialogues
before 1 MTA accepts data this does not impact their efficiency greatly
if they believe that all or most MTAs accepting data place these
messages into inboxes, as setting up and tearing down 10 envelope
dialogues without data costs the spammers no more than the ISPs.
However, the greater the proportion of spam which is
automatically rejected after the data stage, the greater this multiplies
the cost of the spamming operation, for which bandwidth cost is likely to
be a greater proportion of the spammers overall cost.
In practice reputable ISPs are more likely to
be concerned about overall customer satisfaction than the marginal
cost of receiving spam. This issue might put some ISPs into a kind
of prisoners dillemma, but ISPs benefit collectively, the more
spammers they can put out of business.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Richard Kay
2004-01-24 12:37:57 UTC
Permalink
> <tv+***@duh.org> wrote:
> Stopping forgery at the MAIL FROM sender level is pretty cut and dry. I
> don't see how you could then additionally consider the mail to be a
> "forgery". Exactly what additional information are you trying to check!?

At the current stage of SPF deployment and development, I see this
as a promising technology, but something still at the research
stage. I for one am not ready to patch the MTA for my stable
production server in such a manner that this may result in
unknown vulnerabilities, instabilities and false positives. When
the SPF patches have been sufficiently well tested to be adopted
into mainstream Sendmail or similar MTAs, and DNS administrators have
a better understanding of the pattern of legitimate mail origination
for their domains and use of the SPF record syntax etc. my
view on this is very likely to change.

Until then I am a lot more interested in seeing how effectively
SPF tagging pass/fail will be effective in supplementing current
methods of spam rejection, tagging and filtering, the
first stage of which (rejection prior to data) occurs at my MTA
(checking originating domains exist and using a couple of DNSBLs),
and the second stage occuring after the data phase (SpamAssassin and Tagspam,
the latter of which allows me manually to check my spam folder every
few days very quickly to check for false positives. Using Tagspam
to do DNSBL checks after delivery picks up the spam the neccessarily
more accepting secondary MX MTA (which I don't administrate)
allows through. I get about 30 - 40 good messages a day, and
reject about 300 a day at the MTA level and filter out about
another 70 spams a day at the secondary filtering stage.
About 10 spams a day make it into my inbox.

So at this stage I am not ready for any SPF based rejection at the MTA,
but want to do all of this through tagging and procmail rules,
based on later processing, allowing for occasional and quick
manual inspection of my spam folder .

I think there is also an advantage here, in the sense that I
think it probable that SPF
aware spammers will very soon learn how to automatically create
envelope headers that the receiving MTA expects. Forgery, as
I understand this, from
the end user point of view is then about what is in the From: data
header which controls the reply button, and not about what was
in the From envelope sender which most end users won't see in
any case. When reliable MTA rejection
and SPF filtering are both available, I feel a number of people
including myself are likely to want to do both. In the meantime
it would be useful for the standards documents to assist with
either approach, but neither should be mandated.

Richard Kay

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Arik Baratz
2004-01-27 09:43:18 UTC
Permalink
-----Original Message-----
From: Greg Connor [mailto:***@nekodojo.org]
> --Mark Shewmaker <***@primefactor.com> wrote:
> >
> > I did not realize that it was preserved in Return-Path:
>
> Interestingly, my sendmail/pop server shows "Return-Path:" (always at the
> very top) but my Exchange server doesn't show Return-Path at all. Perhaps
> Return-Path: is not consistent. Sort of bad news for SPF if it isn't.

Quoting RFC2821, section 4.4:

When the delivery SMTP server makes the "final delivery" of a
message, it inserts a return-path line at the beginning of the mail
data. This use of return-path is required; mail systems MUST support
it. The return-path line preserves the information in the <reverse-
path> from the MAIL command. Here, final delivery means the message
has left the SMTP environment. Normally, this would mean it had been
delivered to the destination user or an associated mail drop, but in
some cases it may be further processed and transmitted by another
mail system.

Now I don't know what version of Exchange you are using, but my Exchange
2000 SP3 has a Return-Path: on your message to the list:

Return-Path: owner-spf-***@v2.listbox.com

and from cursory glance at my mailbox it seems like they all have a
Return-Path header. You can view it from Outlook by right-clicking
a message, selecting Options and scrolling through the "Internet
headers" message box.

You will NOT see this header for messages that are sent internally
within the Exchange organization, but that's okay, we don't care about
these anyway when dealing with SPF.

-- Arik

**********************************************************************
This email and attachments have been scanned for
potential proprietary or sensitive information leakage.

PortAuthority(TM) Server
Keeping Information Inside
Vidius, Inc.
www.vidius.com
**********************************************************************

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Arik Baratz
2004-01-27 09:47:21 UTC
Permalink
-----Original Message-----
From: wayne [mailto:***@midwestcs.com]

[snip]

> Adding the Return-Path: header is not required and many MTAs don't do
> it. Some use Sender: instead, some many use things like
> Envelope-From:. I forget all the variations.

To me it seems like it IS required. From RFC2821 section 4.4:

When the delivery SMTP server makes the "final delivery" of a
message, it inserts a return-path line at the beginning of the mail
data. This use of return-path is required; mail systems MUST support
it. The return-path line preserves the information in the <reverse-
path> from the MAIL command. Here, final delivery means the message
has left the SMTP environment. Normally, this would mean it had been
delivered to the destination user or an associated mail drop, but in
some cases it may be further processed and transmitted by another
mail system.

If an MTA doesn't support it, it must be reported as a non-compliance
issue to the maintainer.

Anyway, like I replied to the original poster, Exchange 2000 does
support this header. I can do some tests on Exchange 5.5 and get
back to the list if it's ain't there.

-- Arik

**********************************************************************
This email and attachments have been scanned for
potential proprietary or sensitive information leakage.

PortAuthority(TM) Server
Keeping Information Inside
Vidius, Inc.
www.vidius.com
**********************************************************************

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
wayne
2004-01-27 14:33:20 UTC
Permalink
In <***@radon.vidius.co.il> "Arik Baratz" <***@vidius.co.il> writes:

> -----Original Message-----
> From: wayne [mailto:***@midwestcs.com]
>
>> Adding the Return-Path: header is not required and many MTAs don't do
>> it. Some use Sender: instead, some many use things like
>> Envelope-From:. I forget all the variations.
>
> To me it seems like it IS required. From RFC2821 section 4.4:

This is very good news, thanks for the correction!

> If an MTA doesn't support it, it must be reported as a non-compliance
> issue to the maintainer.

Unfortunately, a quick review of NANAS shows that a large percentage
(30-50%) of the posts there do not have a Return-Path: header. So,
whether it is required or not, you can't depend on having it.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Arik Baratz
2004-01-27 18:24:12 UTC
Permalink
-----Original Message-----
From: wayne [mailto:***@midwestcs.com]

> > Well, to make long things short, I think a decent survey needs to be
> > done, by actual people receiving actual mail, and not by looking at
> > NANA*.
>
> Such a survey is going to be hard to do because the Return-Path: is
> only supposed to be added on final delivery. So, you can't just ask
> someone that sends, forwards, or receives lots of email, you have to
> ask lots of different people. NANAS might not be the perfect place to
> check, but it isn't a bad start. I had also suggested the SA spam
> archive and spamarchive.org

How about I fire up a mail message to some of my sysadmin friends,
and ask them to specify which mailer they use and test for the existance
of the header on my behalf, and you do the same. I can test qmail,
postfix and exchange myself:

qmail 1.03 - compliant
postfix 2.0.12 - compliant
Exchange 2000 SP3 - compliant
Exchange 5.5 SP4 - compliant

Folks, if you're interested in helping please check your mail system
- check what MTA is doing local delivery at your site(s) and specify:

A. make and version
B. whether it adds a "Return-Path:" header to messages INBOUND
FROM THE NET

We don't have to reach 100% of the MTAs - we just have to see to it
that there are enough that 90%-95% of the mail gets delivered using
compliant mailers.

-- Arik

**********************************************************************
This email and attachments have been scanned for
potential proprietary or sensitive information leakage.

PortAuthority(TM) Server
Keeping Information Inside
Vidius, Inc.
www.vidius.com
**********************************************************************

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Ernesto Baschny
2004-01-28 08:51:29 UTC
Permalink
On 27 Jan 2004 at 20:24, Arik Baratz wrote:

> How about I fire up a mail message to some of my sysadmin friends,
> and ask them to specify which mailer they use and test for the existance
> of the header on my behalf, and you do the same. I can test qmail,
> postfix and exchange myself:
>
> qmail 1.03 - compliant
> postfix 2.0.12 - compliant
> Exchange 2000 SP3 - compliant
> Exchange 5.5 SP4 - compliant

sendmail 8.12.x - compliant
exim 3.36 - compliant

I guess most conventional MTAs will be "compliant" anyway...

--
Ernesto Baschny <***@baschny.de>
http://www.baschny.de - PGP: http://www.baschny.de/pgp.txt
Sao Paulo/Brasil - Stuttgart/Germany
***@IRCnet - ICQ# 2955403

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
wayne
2004-01-28 12:31:05 UTC
Permalink
In <***@localhost> "Ernesto Baschny" <***@baschny.de> writes:

> On 27 Jan 2004 at 20:24, Arik Baratz wrote:
>
>> How about I fire up a mail message to some of my sysadmin friends,
>> and ask them to specify which mailer they use and test for the existance
>> of the header on my behalf, and you do the same. I can test qmail,
>> postfix and exchange myself:
>>
>> qmail 1.03 - compliant
>> postfix 2.0.12 - compliant
>> Exchange 2000 SP3 - compliant
>> Exchange 5.5 SP4 - compliant
>
> sendmail 8.12.x - compliant
> exim 3.36 - compliant

exim 4.30 -compliant


> I guess most conventional MTAs will be "compliant" anyway...

What matters is how many MTAs that are actually deployed are
complaint.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Justin Mason
2004-01-28 18:01:12 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Ernesto Baschny writes:
> On 27 Jan 2004 at 20:24, Arik Baratz wrote:
>
> > How about I fire up a mail message to some of my sysadmin friends,
> > and ask them to specify which mailer they use and test for the existance
> > of the header on my behalf, and you do the same. I can test qmail,
> > postfix and exchange myself:
> >
> > qmail 1.03 - compliant
> > postfix 2.0.12 - compliant
> > Exchange 2000 SP3 - compliant
> > Exchange 5.5 SP4 - compliant
>
> sendmail 8.12.x - compliant
> exim 3.36 - compliant
>
> I guess most conventional MTAs will be "compliant" anyway...

BTW a very helpful addition to support MUAs and MUA-level filters
getting at the envelope from, would be if this was added to the
Received headers, e.g.

Received: from portent.listbox.com (portent.listbox.com [208.58.1.195])
by amgod.boxhost.net (envelope-from
<owner-spf-discuss(AT)v2.listbox.com>) (Postfix) with ESMTP id
12CE931024F for <jm(AT)jmason.org>; Wed, 28 Jan 2004
08:54:19 +0000 (GMT)

This would allow filters like SpamAssassin to pick up the envelope-from
used at each step of the chain, which is very valuable especially when
intermediate steps tend to rewrite it. (For example, fetchmail makes some
incorrect assumptions, and will add an *incorrect* Return-Path header if
an X-Envelope-From header exists, even from an earlier handover.)

If I recall correctly, some versions of exim seem to support this.

We already have support for this in SpamAssassin -- we detect it
using this regexp matched against Received lines:

/envelope-(?:sender|from)[ =](\S+)\b/

If you know of an MTA package that will add the envelope-sender data to
the Received header, or can write a configuration stanza to add to config
files to do this in any given MTA, please add a note about it on this
page:

http://wiki.spamassassin.org/w/EnvelopeSenderInReceived

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFAF/joQTcbUG5Y7woRApBTAJ4tKhU/h7q1oGj3W6D+n2/bZrZf1wCglBea
G9tYCHkiIi2TM/k9/e4H0ks=
=U1R0
-----END PGP SIGNATURE-----

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Julian Mehnle
2004-01-28 13:26:59 UTC
Permalink
Arik Baratz [***@vidius.co.il] wrote:
> How about I fire up a mail message to some of my sysadmin friends,
> and ask them to specify which mailer they use and test for the existance
> of the header on my behalf, and you do the same. I can test qmail,
> postfix and exchange myself:
>
> qmail 1.03 - compliant
> postfix 2.0.12 - compliant
> Exchange 2000 SP3 - compliant
> Exchange 5.5 SP4 - compliant

Courier 0.42.2 - compliant

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Arik Baratz
2004-01-28 18:38:44 UTC
Permalink
-----Original Message-----
From: Justin Mason [mailto:***@jmason.org]

Ernesto Baschny writes:

[snip]

> BTW a very helpful addition to support MUAs and MUA-level filters
> getting at the envelope from, would be if this was added to the
> Received headers, e.g.
>
> Received: from portent.listbox.com (portent.listbox.com [208.58.1.195])
> by amgod.boxhost.net (envelope-from
> <owner-spf-discuss(AT)v2.listbox.com>) (Postfix) with ESMTP id
> 12CE931024F for <jm(AT)jmason.org>; Wed, 28 Jan 2004
> 08:54:19 +0000 (GMT)

I am so strongly against that. There is already a standard header to do
just that! Why reinvent it?

> This would allow filters like SpamAssassin to pick up the envelope-from
> used at each step of the chain, which is very valuable especially when
> intermediate steps tend to rewrite it. (For example, fetchmail makes some
> incorrect assumptions, and will add an *incorrect* Return-Path header if
> an X-Envelope-From header exists, even from an earlier handover.)

>From RFC-2821 section 4.4: (I changed the uppercase sentence to upercase)

When the delivery SMTP server makes the "final delivery" of a
message, it inserts a return-path line AT THE BEGINNING OF THE MAIL
DATA. This use of return-path is required; mail systems MUST support
it. The return-path line preserves the information in the <reverse-
path> from the MAIL command. Here, final delivery means the message
has left the SMTP environment. Normally, this would mean it had been
delivered to the destination user or an associated mail drop, but in
some cases it may be further processed and transmitted by another
mail system.

There can't be an incorrect header, because you know YOUR system adds
a true one and it is ths FIRST (in fact the first line in many cases).

Why do we have to invent a standard when one exists, and it's good
enough? All we need is to ensure compliance by the current array of
popular MTAs, and that's that.

I suggest anyone suggesting anything added to MTAs to first READ THE
F'ING RFCS and check if their suggestion is at all relevant. This is the
third time I quote the same paragraph from the same RFC in this thread.
Don't make me do that again, please.

-- Arik




**********************************************************************
This email and attachments have been scanned for
potential proprietary or sensitive information leakage.

PortAuthority(TM) Server
Keeping Information Inside
Vidius, Inc.
www.vidius.com
**********************************************************************

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Justin Mason
2004-01-28 19:12:26 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Arik Baratz writes:
>I am so strongly against that. There is already a standard header to do
>just that! Why reinvent it?

Reread this paragraph:

>> This would allow filters like SpamAssassin to pick up the envelope-from
>> used at each step of the chain, which is very valuable especially when
>> intermediate steps tend to rewrite it. (For example, fetchmail makes some
>> incorrect assumptions, and will add an *incorrect* Return-Path header if
>> an X-Envelope-From header exists, even from an earlier handover.)

Key phrase: "the envelope-from used at each step of the chain".

In this situation:

SENDER -> FORWARDER -> RECIPIENT

There is currently no way to find out what the env from was at FORWARDER
if it has been rewritten -- as it would be for a mailing list or an
SRS-compliant forwarder.

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFAGAmaQTcbUG5Y7woRAol+AJ9/0U2iEXD3GH3XQXnTJwMvgpojaQCeOGkf
zTcpKZ1TfYQx5B78qROMx0I=
=+BBV
-----END PGP SIGNATURE-----

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Arik Baratz
2004-01-29 00:33:07 UTC
Permalink
Justin Mason wrote:

>>>This would allow filters like SpamAssassin to pick up the envelope-from
>>>used at each step of the chain, which is very valuable especially when
>>>intermediate steps tend to rewrite it. (For example, fetchmail makes some
>>>incorrect assumptions, and will add an *incorrect* Return-Path header if
>>>an X-Envelope-From header exists, even from an earlier handover.)
>>>
>>>
>
>Key phrase: "the envelope-from used at each step of the chain".
>
>
I'm sorry, I was in a hurry I didn't read your entire text. I looked at
the wiki and I saw you describe the Return-Path header as 'Traditional',
in close proximity to the non-standard X-Envelope-From. There's nothing
traditional about it. It's an implemented standard.

Just two things:

A. If SPF is not used, what would prevent the would be spammer from
forging these headers?
B. If SPF is used, why does SpamAssasain care about this header?

-- Arik



-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Justin Mason
2004-01-29 03:07:44 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Arik Baratz writes:
>Justin Mason wrote:
>>>>This would allow filters like SpamAssassin to pick up the envelope-from
>>>>used at each step of the chain, which is very valuable especially when
>>>>intermediate steps tend to rewrite it. (For example, fetchmail makes some
>>>>incorrect assumptions, and will add an *incorrect* Return-Path header if
>>>>an X-Envelope-From header exists, even from an earlier handover.)
>>
>>Key phrase: "the envelope-from used at each step of the chain".
>>
>I'm sorry, I was in a hurry I didn't read your entire text. I looked at
>the wiki and I saw you describe the Return-Path header as 'Traditional',
>in close proximity to the non-standard X-Envelope-From. There's nothing
>traditional about it. It's an implemented standard.
>
>Just two things:
>
>A. If SPF is not used, what would prevent the would be spammer from
>forging these headers?

SpamAssassin already uses code to address this, by determining
(both by user config and automatically where possible) how far
back it can "trust" in the Received chain. In this case, for
example, I may trust the relay for CPAN.org, so that I can
use the recorded env-from from that hop when I receive
mail addressed to <jmason /at/ cpan.org>.

>B. If SPF is used, why does SpamAssasain care about this header?

So it can perform SPF checks reliably for more than just the
"SpamAssassin running on external MX" case ;)

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFAGHkAQTcbUG5Y7woRAuSKAJ4pnDQslKp2MmtK3ULCEnfwMdkBrgCgwaFi
CmLQXsIGkYMRD1br+2cEo5g=
=zvtc
-----END PGP SIGNATURE-----

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Latest draft at http://spf.pobox.com/draft-mengwong-spf-02.9.4.txt
Wiki: http://spfwiki.infinitepenguins.net/pmwiki.php/SenderPermittedFrom/HomePage
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-***@v2.listbox.com
Continue reading on narkive:
Loading...