OpenSPF Why Page and SPF record types

[Richard Lawley]

Hi,

I'm not sure if anyone on here is responsible for or involved with the
OpenSPF WHY page.

I just spent quite some time tracking down a problem with SPF records on
one of my domains, which ended up being down to the DNS software my DNS
host uses serving a synthesised SPF type record. This was invisible
through their editor, and is also hard to query since support for the
record type was not in the version of dig which I was initially using, nor
does nslookup support it in Windows. The problem also compounded by them
serving different synthesised results from both of the nameservers, one of
them ending in -all and the other ~all.

This is clearly a niche situation - for a message to be bounced it had to
be checked by an SPF implementation that took SPF-type records instead of
TXT-type records, and it had to have been served by the DNS server with the
-all record. However, less of a niche situation would be where SPF and TXT
records both exist but do not match.

One of the bounce messages has directed me to the OpenSPF WHY page, which
was showing me that the message didn't match my SPF record, but that it
shouldn't have stopped the message (presumably it had hit the server with
the ~all record). What I would like to suggest is that the record checker
prints the contents of the SPF record it retrieved (and ideally the type of
record it is!) in order to make it more obvious what was going on.

Additional diagnostic steps could potentially be added, such as showing
that conflicting SPF and TXT records exist, but my first suggestion would
have helped me solve this a lot. Just hoping that this can help someone
else in a similar situation!

Regards,

Richard



-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/6959934-50ec8f89
Modify Your Subscription: https://www.listbox.com/member/?member_id=6959934&id_secret=6959934-b7c4528d
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=6959934&id_secret=6959934-edadf31a&post_id=20130801041958:23FDF6DA-FA83-11E2-9291-F76E11191F9C
Powered by Listbox: http://www.listbox.com

[alan]

At 09:19 01/08/2013 Thursday, Richard Lawley wrote:
>Hi,
>
>I'm not sure if anyone on here is responsible for or involved with the OpenSPF WHY page.

i am not the maintainer
but i can clearly see in
http://www.openspf.org/Why/API

that your request would be impossible
as it would require at least 4 more fields

that would require both modification to spf-librarys (to provide the information to users)
and the website to allow users to pass it on

a did i lookup spf or txt
b ip of your dns server i looked up
c "full strng of the spf record I obtained"

and none of these pieces of information are available to the person constructing the link

as my MTA would be the software making the call to the spf-library
supplying mfrom or hostname, ip-recieved-from
and receiving a reply of pass/fall/neutral/hardfail/softfail + "spf-writers-exp-txt" as output

and then constructing the url knowing only the data above + my own hostname and that I'm using spfv1
(as no one sane allows the use of M$-sender-ID aka v2)


>I just spent quite some time tracking down a problem with SPF records on one of my domains, which ended up being down to the DNS software my DNS host uses serving a synthesised SPF type record. This was invisible through their editor, and is also hard to query since support for the record type was not in the version of dig which I was initially using, nor does nslookup support it in Windows. The problem also compounded by them serving different synthesised results from both of the nameservers, one of them ending in -all and the other ~all.

I would suggest this is such an uncommon problem as to not warrent a solution
(simply drop any provider crazy enough to, A use software that adds anything to your DNS without your consent/knowledge, B neither maintain the software so both servers are consistent in results, C fails to inform you of the software clearly, the Internet is a big place with many providers do not support anyone incapable of doing their job well and responsibly, and DNS is too key to leave in the hands of incompetents)

>This is clearly a niche situation - for a message to be bounced it had to be checked by an SPF implementation that took SPF-type records instead of TXT-type records

there are many (actually many will first check spf, then if none check txt)
those many are (luckily for you) limited to the capabilities of the resolving DNS servers they use
(as many of those resolvers do-not handle spf, so the library then goes on to txt after it gets an error in response to its initial spf lookup)

>, and it had to have been served by the DNS server with the -all record. However, less of a niche situation would be where SPF and TXT records both exist but do not match.

in either case the first returned spf will always be the one used
(as one of the things any library attempts to do is limit the number of queries it will make, so if it does spf and txt, it will only try the second one if it does not get a response from the first)

It is/was safely assumed that if spf and txt records both exist the admin would have sole responsibility for ensuring both were accurate, error free and identical.

If you could name the provider and what dns tool/software is doing this heinous act it would be appreciated, as sort of providers/software we could all do with being warned against accidentally using


>One of the bounce messages has directed me to the OpenSPF WHY page, which was showing me that the message didn't match my SPF record, but that it shouldn't have stopped the message (presumably it had hit the server with the ~all record). What I would like to suggest is that the record checker prints the contents of the SPF record it retrieved (and ideally the type of record it is!) in order to make it more obvious what was going on.
>
>Additional diagnostic steps could potentially be added, such as showing that conflicting SPF and TXT records exist, but my first suggestion would have helped me solve this a lot. Just hoping that this can help someone else in a similar situation!
>
>Regards,
>
>Richard
>
>Sender Policy Framework: <http://www.openspf.net>http://www.openspf.net
>Modify Your Subscription: <http://www.listbox.com/member/>http://www.listbox.com/member/
><https://www.listbox.com/member/archive/735/=now>Archives<https://www.listbox.com/member/archive/rss/735/13124949-ec5a0568> | <https://www.listbox.com/member/?&>Modify Your Subscription | <https://www.listbox.com/unsubscribe/?&&post_id=20130801041958:23FDF6DA-FA83-11E2-9291-F76E11191F9C>Unsubscribe Now<http://www.listbox.com>

[Richard Lawley]

Alan,

Actually my suggestion requires NO changes to the API. It was a suggestion
that the WHY diagnostic page would display the record it used in displaying
the message - nothing to do with the data being passed in the URL, or any
changes to the MTA. The WHY page does its own check using the parameters
supplied in the URL and displays the results. My suggestion is just to add
extra information to these results - the value of the SPF/TXT record
retrieved in order to do the check.

For example, I went to the WHY page using the link from my bounce before,
and it said there was a problem. If I visit the same link now after fixing
my DNS records, it says that there is no reason for a bounce - the page
looks up my SPF record on the fly.

I'm aware that SPF records will be used if they exist before TXT records,
and I was not suggesting that SPF libraries should behave any differently,
and definitely should not do more DNS lookups than necessary. Diagnostic
tools, however, are a different matter and that's where I was suggesting
the change be made. The WHY page already returns more information than
your SPF library would return to the MTA (e.g. enough information to
explain the ~all/-all setting on the record it examined).

The DNS server software concerned is Simple DNS, which has a feature where
it will synthesise SPF records if none are present, and another to
synthesis a TXT record to match a SPF record if only one is present (though
not the other way around). The provider previously did not support editing
of SPF records, but in response to my support request has enabled this
request. The problem therefore is no longer an issue for me, but I was
trying to make suggestions which may help others in the same situation.

Regards,

Richard


On 1 August 2013 10:09, alan <***@alandoherty.net> wrote:

> At 09:19 01/08/2013 Thursday, Richard Lawley wrote:
> >Hi,
> >
> >I'm not sure if anyone on here is responsible for or involved with the
> OpenSPF WHY page.
>
> i am not the maintainer
> but i can clearly see in
> http://www.openspf.org/Why/API
>
> that your request would be impossible
> as it would require at least 4 more fields
>
> that would require both modification to spf-librarys (to provide the
> information to users)
> and the website to allow users to pass it on
>
> a did i lookup spf or txt
> b ip of your dns server i looked up
> c "full strng of the spf record I obtained"
>
> and none of these pieces of information are available to the person
> constructing the link
>
> as my MTA would be the software making the call to the spf-library
> supplying mfrom or hostname, ip-recieved-from
> and receiving a reply of pass/fall/neutral/hardfail/softfail +
> "spf-writers-exp-txt" as output
>
> and then constructing the url knowing only the data above + my own
> hostname and that I'm using spfv1
> (as no one sane allows the use of M$-sender-ID aka v2)
>
>
> >I just spent quite some time tracking down a problem with SPF records on
> one of my domains, which ended up being down to the DNS software my DNS
> host uses serving a synthesised SPF type record. This was invisible
> through their editor, and is also hard to query since support for the
> record type was not in the version of dig which I was initially using, nor
> does nslookup support it in Windows. The problem also compounded by them
> serving different synthesised results from both of the nameservers, one of
> them ending in -all and the other ~all.
>
> I would suggest this is such an uncommon problem as to not warrent a
> solution
> (simply drop any provider crazy enough to, A use software that adds
> anything to your DNS without your consent/knowledge, B neither maintain the
> software so both servers are consistent in results, C fails to inform you
> of the software clearly, the Internet is a big place with many providers do
> not support anyone incapable of doing their job well and responsibly, and
> DNS is too key to leave in the hands of incompetents)
>
> >This is clearly a niche situation - for a message to be bounced it had to
> be checked by an SPF implementation that took SPF-type records instead of
> TXT-type records
>
> there are many (actually many will first check spf, then if none check txt)
> those many are (luckily for you) limited to the capabilities of the
> resolving DNS servers they use
> (as many of those resolvers do-not handle spf, so the library then goes on
> to txt after it gets an error in response to its initial spf lookup)
>
> >, and it had to have been served by the DNS server with the -all record.
> However, less of a niche situation would be where SPF and TXT records both
> exist but do not match.
>
> in either case the first returned spf will always be the one used
> (as one of the things any library attempts to do is limit the number of
> queries it will make, so if it does spf and txt, it will only try the
> second one if it does not get a response from the first)
>
> It is/was safely assumed that if spf and txt records both exist the admin
> would have sole responsibility for ensuring both were accurate, error free
> and identical.
>
> If you could name the provider and what dns tool/software is doing this
> heinous act it would be appreciated, as sort of providers/software we could
> all do with being warned against accidentally using
>
>
> >One of the bounce messages has directed me to the OpenSPF WHY page, which
> was showing me that the message didn't match my SPF record, but that it
> shouldn't have stopped the message (presumably it had hit the server with
> the ~all record). What I would like to suggest is that the record checker
> prints the contents of the SPF record it retrieved (and ideally the type of
> record it is!) in order to make it more obvious what was going on.
> >
> >Additional diagnostic steps could potentially be added, such as showing
> that conflicting SPF and TXT records exist, but my first suggestion would
> have helped me solve this a lot. Just hoping that this can help someone
> else in a similar situation!
> >
> >Regards,
> >
> >Richard
> >
> >Sender Policy Framework: <http://www.openspf.net>http://www.openspf.net
> >Modify Your Subscription: <http://www.listbox.com/member/>
> http://www.listbox.com/member/
> ><https://www.listbox.com/member/archive/735/=now>Archives<
> https://www.listbox.com/member/archive/rss/735/13124949-ec5a0568> | <
> https://www.listbox.com/member/?&>Modify Your Subscription | <
> https://www.listbox.com/unsubscribe/?&&post_id=20130801041958:23FDF6DA-FA83-11E2-9291-F76E11191F9C>Unsubscribe
> Now<http://www.listbox.com>
>
>
>
> -------------------------------------------
> Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
> Modify Your Subscription: http://www.listbox.com/member/ [
> http://www.listbox.com/member/]
>
> Archives: https://www.listbox.com/member/archive/735/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/735/24836610-4f4eb4b5
> Modify Your Subscription:
> https://www.listbox.com/member/?&
> Unsubscribe Now:
> https://www.listbox.com/unsubscribe/?&&post_id=20130801051108:4B237CEC-FA8A-11E2-B638-85190246559F
> Powered by Listbox: http://www.listbox.com
>



-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/6959934-50ec8f89
Modify Your Subscription: https://www.listbox.com/member/?member_id=6959934&id_secret=6959934-b7c4528d
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=6959934&id_secret=6959934-edadf31a&post_id=20130801063922:9DB4D2D8-FA96-11E2-A370-DC1D38637AE9
Powered by Listbox: http://www.listbox.com

[alan]

the why page is not a diagnostic tool

its a service for mail server admins to be able to point users at an existing error page "with usefull advice" rather than having to produce their own explanatory text and documentation

for receivers of bounces, not writers of spf records

the tools are here
http://www.openspf.org/Tools

and the linked http://www.kitterman.com/spf/validate.html
would have shown/displayed the errant SPF/99 record

At 11:39 01/08/2013 Thursday, Richard Lawley wrote:
>Alan,
>
>Actually my suggestion requires NO changes to the API. It was a suggestion that the WHY diagnostic page would display the record it used in displaying the message - nothing to do with the data being passed in the URL, or any changes to the MTA. The WHY page does its own check using the parameters supplied in the URL and displays the results. My suggestion is just to add extra information to these results - the value of the SPF/TXT record retrieved in order to do the check.
>
>For example, I went to the WHY page using the link from my bounce before, and it said there was a problem. If I visit the same link now after fixing my DNS records, it says that there is no reason for a bounce - the page looks up my SPF record on the fly.
>
>I'm aware that SPF records will be used if they exist before TXT records, and I was not suggesting that SPF libraries should behave any differently, and definitely should not do more DNS lookups than necessary. Diagnostic tools, however, are a different matter and that's where I was suggesting the change be made. The WHY page already returns more information than your SPF library would return to the MTA (e.g. enough information to explain the ~all/-all setting on the record it examined).
>
>The DNS server software concerned is Simple DNS, which has a feature where it will synthesise SPF records if none are present, and another to synthesis a TXT record to match a SPF record if only one is present (though not the other way around). The provider previously did not support editing of SPF records, but in response to my support request has enabled this request. The problem therefore is no longer an issue for me, but I was trying to make suggestions which may help others in the same situation.

the best way appears to be to avoid providers using simple DNS

no provider should ever publish a SPF (or any other DNS entry) without explicit decision to do so made by the domain owner/admin

as a provider myself I would consider it an actionable, and liability producing offence for me/staff or software to ignore this rule


>Regards,
>
>Richard
>
>
>On 1 August 2013 10:09, alan <<mailto:***@alandoherty.net>***@alandoherty.net> wrote:
>At 09:19 01/08/2013 Thursday, Richard Lawley wrote:
>>Hi,
>>
>>I'm not sure if anyone on here is responsible for or involved with the OpenSPF WHY page.
>
>i am not the maintainer
>but i can clearly see in
><http://www.openspf.org/Why/API>http://www.openspf.org/Why/API
>
>that your request would be impossible
>as it would require at least 4 more fields
>
>that would require both modification to spf-librarys (to provide the information to users)
>and the website to allow users to pass it on
>
>a did i lookup spf or txt
>b ip of your dns server i looked up
>c "full strng of the spf record I obtained"
>
>and none of these pieces of information are available to the person constructing the link
>
>as my MTA would be the software making the call to the spf-library
>supplying mfrom or hostname, ip-recieved-from
>and receiving a reply of pass/fall/neutral/hardfail/softfail + "spf-writers-exp-txt" as output
>
>and then constructing the url knowing only the data above + my own hostname and that I'm using spfv1
>(as no one sane allows the use of M$-sender-ID aka v2)
>
>
>>I just spent quite some time tracking down a problem with SPF records on one of my domains, which ended up being down to the DNS software my DNS host uses serving a synthesised SPF type record. This was invisible through their editor, and is also hard to query since support for the record type was not in the version of dig which I was initially using, nor does nslookup support it in Windows. The problem also compounded by them serving different synthesised results from both of the nameservers, one of them ending in -all and the other ~all.
>
>I would suggest this is such an uncommon problem as to not warrent a solution
>(simply drop any provider crazy enough to, A use software that adds anything to your DNS without your consent/knowledge, B neither maintain the software so both servers are consistent in results, C fails to inform you of the software clearly, the Internet is a big place with many providers do not support anyone incapable of doing their job well and responsibly, and DNS is too key to leave in the hands of incompetents)
>
>>This is clearly a niche situation - for a message to be bounced it had to be checked by an SPF implementation that took SPF-type records instead of TXT-type records
>
>there are many (actually many will first check spf, then if none check txt)
>those many are (luckily for you) limited to the capabilities of the resolving DNS servers they use
>(as many of those resolvers do-not handle spf, so the library then goes on to txt after it gets an error in response to its initial spf lookup)
>
>>, and it had to have been served by the DNS server with the -all record. However, less of a niche situation would be where SPF and TXT records both exist but do not match.
>
>in either case the first returned spf will always be the one used
>(as one of the things any library attempts to do is limit the number of queries it will make, so if it does spf and txt, it will only try the second one if it does not get a response from the first)
>
>It is/was safely assumed that if spf and txt records both exist the admin would have sole responsibility for ensuring both were accurate, error free and identical.
>
>If you could name the provider and what dns tool/software is doing this heinous act it would be appreciated, as sort of providers/software we could all do with being warned against accidentally using
>
>
>>One of the bounce messages has directed me to the OpenSPF WHY page, which was showing me that the message didn't match my SPF record, but that it shouldn't have stopped the message (presumably it had hit the server with the ~all record). What I would like to suggest is that the record checker prints the contents of the SPF record it retrieved (and ideally the type of record it is!) in order to make it more obvious what was going on.
>>
>>Additional diagnostic steps could potentially be added, such as showing that conflicting SPF and TXT records exist, but my first suggestion would have helped me solve this a lot. Just hoping that this can help someone else in a similar situation!
>>
>>Regards,
>>
>>Richard
>>
>>Sender Policy Framework: <<http://www.openspf.net>http://www.openspf.net>http://www.openspf.net
>>Modify Your Subscription: <<http://www.listbox.com/member/>http://www.listbox.com/member/>http://www.listbox.com/member/
>><<https://www.listbox.com/member/archive/735/=now>https://www.listbox.com/member/archive/735/=now>Archives<<https://www.listbox.com/member/archive/rss/735/13124949-ec5a0568>https://www.listbox.com/member/archive/rss/735/13124949-ec5a0568> | <https://www.listbox.com/member/?&>Modify Your Subscription | <<https://www.listbox.com/unsubscribe/?&&post_id=20130801041958:23FDF6DA-FA83-11E2-9291-F76E11191F9C>https://www.listbox.com/unsubscribe/?&&post_id=20130801041958:23FDF6DA-FA83-11E2-9291-F76E11191F9C>Unsubscribe Now<<http://www.listbox.com>http://www.listbox.com>
>
>
>
>-------------------------------------------
>Sender Policy Framework: <http://www.openspf.net>http://www.openspf.net [http://www.openspf.net]
>Modify Your Subscription: <http://www.listbox.com/member/>http://www.listbox.com/member/ [http://www.listbox.com/member/]
>
>Archives: <https://www.listbox.com/member/archive/735/=now>https://www.listbox.com/member/archive/735/=now
>RSS Feed: <https://www.listbox.com/member/archive/rss/735/24836610-4f4eb4b5>https://www.listbox.com/member/archive/rss/735/24836610-4f4eb4b5
>Modify Your Subscription: <https://www.listbox.com/member/?&>https://www.listbox.com/member/?&
>Unsubscribe Now: <https://www.listbox.com/unsubscribe/?&&post_id=20130801051108:4B237CEC-FA8A-11E2-B638-85190246559F>https://www.listbox.com/unsubscribe/?&&post_id=20130801051108:4B237CEC-FA8A-11E2-B638-85190246559F
>Powered by Listbox: <http://www.listbox.com>http://www.listbox.com
>
>
>Sender Policy Framework: <http://www.openspf.net>http://www.openspf.net
>Modify Your Subscription: <http://www.listbox.com/member/>http://www.listbox.com/member/
><https://www.listbox.com/member/archive/735/=now>Archives<https://www.listbox.com/member/archive/rss/735/13124949-ec5a0568> | <https://www.listbox.com/member/?&>Modify Your Subscription | <https://www.listbox.com/unsubscribe/?&&post_id=20130801063922:9DB4D2D8-FA96-11E2-A370-DC1D38637AE9>Unsubscribe Now<http://www.listbox.com>

[Alessandro Vesely]

On Thu 01/Aug/2013 15:28:13 +0200 alan wrote:
> At 11:39 01/08/2013 Thursday, Richard Lawley wrote:
>>
>> The DNS server software concerned is Simple DNS, which has a
>> feature where it will synthesise SPF records if none are present,
>> and another to synthesis a TXT record to match a SPF record if only
>> one is present (though not the other way around). The provider
>> previously did not support editing of SPF records, but in response
>> to my support request has enabled this request. The problem
>> therefore is no longer an issue for me, but I was trying to make
>> suggestions which may help others in the same situation.
>
> the best way appears to be to avoid providers using simple DNS

I beg to differ. Those SPF features look compelling. Check
http://support.simpledns.com/KB/a81/configuring-spf-sender-policy-framework-records.aspx

> no provider should ever publish a SPF (or any other DNS entry)
> without explicit decision to do so made by the domain owner/admin

Absolutely! However, the point seems to be that providers should
understand their users' requests, and match them to their DNS software
features appropriately. That may be non trivial when those features are
not as simple as their name may suggest.