Mailing List Archive

def_whitelist_auth inconsistencies
I can't figure this one out so I'll throw it out.

When receiving mail from the address bbc@e.bbcmail.co.uk

def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
whitelist_from_dkim *@*.bbcmail.co.uk # is whitelisted

Now I thought this a bit odd since the docs say:

Using whitelist_auth is roughly equivalent to specifying duplicate
whitelist_from_spf, whitelist_from_dk, and whitelist_from_dkim lines for
each of the addresses specified.

But it is only "roughly equivalent" so perhaps it doesn't like the
wildcard subdomain syntax.

And then I saw info@fun.stubhub.co.uk be whitelisted by:

def_whitelist_auth *@*.stubhub.co.uk

So presumably whitelist_auth will accept a wildcard subdomain, but not for
the bbcemail.co.uk domain...

I've been running all kinds of variations through and I can't really see
why one is whitelisted and the other isn't. There's no obvious difference
(to me) between these addresses, DKIM signatures, or whitelisting rules,
yet still they behave differently.

Can anyone spot my mistake?
Re: def_whitelist_auth inconsistencies [ In reply to ]
On 23.03.16 11:56, Kevin Golding wrote:
>I can't figure this one out so I'll throw it out.
>
>When receiving mail from the address bbc@e.bbcmail.co.uk
>
>def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted

well, it should be, but only -15 points, sice it's def_whitelist and not
whitelist. check for this carefully...

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 12:30:43 -0000, Matus UHLAR - fantomas
<uhlar@fantomas.sk> wrote:

> On 23.03.16 11:56, Kevin Golding wrote:
>> I can't figure this one out so I'll throw it out.
>>
>> When receiving mail from the address bbc@e.bbcmail.co.uk
>>
>> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
>
> well, it should be, but only -15 points, sice it's def_whitelist and not
> whitelist. check for this carefully...
>

Alas, definitely not:

Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by e.bbcmail.co.uk,
author bbc@e.bbcmail.co.uk, no valid matches
Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk, not in
any dkim whitelist
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 13:45:29 -0000
Kevin Golding wrote:

> On Wed, 23 Mar 2016 12:30:43 -0000, Matus UHLAR - fantomas
> <uhlar@fantomas.sk> wrote:
>
> > On 23.03.16 11:56, Kevin Golding wrote:
> >> I can't figure this one out so I'll throw it out.
> >>
> >> When receiving mail from the address bbc@e.bbcmail.co.uk
> >>
> >> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
> >
> > well, it should be, but only -15 points, sice it's def_whitelist
> > and not whitelist. check for this carefully...
> >
>
> Alas, definitely not:
>
> Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
> e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
> Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk,
> not in any dkim whitelist


do you have debug where bbc@e.bbcmail.co.uk both matches and fails on
on the pattern *@*.bbcmail.co.uk on the _same_ scan?
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 14:04:03 -0000, RW <rwmaillists@googlemail.com> wrote:

> On Wed, 23 Mar 2016 13:45:29 -0000
> Kevin Golding wrote:
>
>> On Wed, 23 Mar 2016 12:30:43 -0000, Matus UHLAR - fantomas
>> <uhlar@fantomas.sk> wrote:
>>
>> > On 23.03.16 11:56, Kevin Golding wrote:
>> >> I can't figure this one out so I'll throw it out.
>> >>
>> >> When receiving mail from the address bbc@e.bbcmail.co.uk
>> >>
>> >> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
>> >
>> > well, it should be, but only -15 points, sice it's def_whitelist
>> > and not whitelist. check for this carefully...
>> >
>>
>> Alas, definitely not:
>>
>> Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
>> e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
>> Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk,
>> not in any dkim whitelist
>
>
> do you have debug where bbc@e.bbcmail.co.uk both matches and fails on
> on the pattern *@*.bbcmail.co.uk on the _same_ scan?
>

I assume you mean can I show it matching on the same message? Since if I
have both enabled on a scan it's just a case of the first to hit will be
displayed I believe, and any failures won't show at all. With both enabled
I get:

Mar 23 15:26:55.945 [27188] dbg: dkim: VALID author domain signature by
e.bbcmail.co.uk, MATCHES whitelist_from_dkim
(?^i:^.*\@.*\.bbcmail\.co\.uk$)
Mar 23 15:26:55.945 [27188] dbg: dkim: author bbc@e.bbcmail.co.uk,
WHITELISTED by whitelist_from_dkim/e.bbcmail.co.uk

And def_whitelist_auth will match on both the full address:

Mar 23 11:21:52.535 [15896] dbg: dkim: VALID author domain signature by
e.bbcmail.co.uk, MATCHES def_whitelist_auth ^bbc\@e\.bbcmail\.co\.uk$
Mar 23 11:21:52.535 [15896] dbg: dkim: author bbc@e.bbcmail.co.uk,
WHITELISTED by def_whitelist_auth/e.bbcmail.co.uk

And also if I only wildcard the user part:

Mar 23 11:25:12.688 [16086] dbg: dkim: VALID author domain signature by
e.bbcmail.co.uk, MATCHES def_whitelist_auth ^.*\@e\.bbcmail\.co\.uk$
Mar 23 11:25:12.688 [16086] dbg: dkim: author bbc@e.bbcmail.co.uk,
WHITELISTED by def_whitelist_auth/e.bbcmail.co.uk

It only fails on def_whitelist_auth *@*.bbcmail.co.uk

Same message each time, the only variation on the scans are the whitelist
settings.
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 14:36:08 -0000
Kevin Golding wrote:

> On Wed, 23 Mar 2016 14:04:03 -0000, RW <rwmaillists@googlemail.com>
> wrote:
>
> > On Wed, 23 Mar 2016 13:45:29 -0000
> > Kevin Golding wrote:
> >
> >> On Wed, 23 Mar 2016 12:30:43 -0000, Matus UHLAR - fantomas
> >> <uhlar@fantomas.sk> wrote:
> >>
> >> > On 23.03.16 11:56, Kevin Golding wrote:
> >> >> I can't figure this one out so I'll throw it out.
> >> >>
> >> >> When receiving mail from the address bbc@e.bbcmail.co.uk
> >> >>
> >> >> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
> >> >
> >> > well, it should be, but only -15 points, sice it's def_whitelist
> >> > and not whitelist. check for this carefully...
> >> >
> >>
> >> Alas, definitely not:
> >>
> >> Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
> >> e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
> >> Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk,
> >> not in any dkim whitelist
> >
> >
> > do you have debug where bbc@e.bbcmail.co.uk both matches and fails
> > on on the pattern *@*.bbcmail.co.uk on the _same_ scan?
> >
>
> I assume you mean can I show it matching on the same message? Since
> if I have both enabled on a scan it's just a case of the first to hit
> will be displayed I believe, and any failures won't show at all. With
> both enabled I get:

The original post had:

def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
whitelist_from_dkim *@*.bbcmail.co.uk # is whitelisted

which I presumed meant that both lines were present in your config and
mail from bbc@e.bbcmail.co.uk was hitting USER_IN_DKIM_WHITELIST, but
not USER_IN_DEF_DKIM_WL.

I wanted to see the debug for the case where one rule works as
expected and the other doesn't - and it's important to see this on
the same mail.

The first two debug entries below show both def and normal whitelist
hits working properly, but they are confused by a *@bbcmail.co.uk
whitelisting entry which you say fixes the problem. Below that you
have a def_whitelist_auth match with *@bbcmail.co.uk. All that you are
illustrating here is that *@bbcmail.co.uk works as expected, not that
*@*bbcmail.co.uk fails on anything.



> Mar 23 15:26:55.945 [27188] dbg: dkim: VALID author domain signature
> by e.bbcmail.co.uk, MATCHES whitelist_from_dkim
> (?^i:^.*\@.*\.bbcmail\.co\.uk$)
> Mar 23 15:26:55.945 [27188] dbg: dkim: author bbc@e.bbcmail.co.uk,
> WHITELISTED by whitelist_from_dkim/e.bbcmail.co.uk
>
> And def_whitelist_auth will match on both the full address:
>
> Mar 23 11:21:52.535 [15896] dbg: dkim: VALID author domain signature
> by e.bbcmail.co.uk, MATCHES def_whitelist_auth
> ^bbc\@e\.bbcmail\.co\.uk$ Mar 23 11:21:52.535 [15896] dbg: dkim:
> author bbc@e.bbcmail.co.uk, WHITELISTED by
> def_whitelist_auth/e.bbcmail.co.uk
>
> And also if I only wildcard the user part:
>
> Mar 23 11:25:12.688 [16086] dbg: dkim: VALID author domain signature
> by e.bbcmail.co.uk, MATCHES def_whitelist_auth
> ^.*\@e\.bbcmail\.co\.uk$ Mar 23 11:25:12.688 [16086] dbg: dkim:
> author bbc@e.bbcmail.co.uk, WHITELISTED by
> def_whitelist_auth/e.bbcmail.co.uk
>
> It only fails on def_whitelist_auth *@*.bbcmail.co.uk
>
> Same message each time, the only variation on the scans are the
> whitelist settings.
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 15:38:33 -0000, RW <rwmaillists@googlemail.com> wrote:

> On Wed, 23 Mar 2016 14:36:08 -0000
> Kevin Golding wrote:
>
>> On Wed, 23 Mar 2016 14:04:03 -0000, RW <rwmaillists@googlemail.com>
>> wrote:
>>
>> > On Wed, 23 Mar 2016 13:45:29 -0000
>> > Kevin Golding wrote:
>> >
>> >> On Wed, 23 Mar 2016 12:30:43 -0000, Matus UHLAR - fantomas
>> >> <uhlar@fantomas.sk> wrote:
>> >>
>> >> > On 23.03.16 11:56, Kevin Golding wrote:
>> >> >> I can't figure this one out so I'll throw it out.
>> >> >>
>> >> >> When receiving mail from the address bbc@e.bbcmail.co.uk
>> >> >>
>> >> >> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
>> >> >
>> >> > well, it should be, but only -15 points, sice it's def_whitelist
>> >> > and not whitelist. check for this carefully...
>> >> >
>> >>
>> >> Alas, definitely not:
>> >>
>> >> Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
>> >> e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
>> >> Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk,
>> >> not in any dkim whitelist
>> >
>> >
>> > do you have debug where bbc@e.bbcmail.co.uk both matches and fails
>> > on on the pattern *@*.bbcmail.co.uk on the _same_ scan?
>> >
>>
>> I assume you mean can I show it matching on the same message? Since
>> if I have both enabled on a scan it's just a case of the first to hit
>> will be displayed I believe, and any failures won't show at all. With
>> both enabled I get:
>
> The original post had:
>
> def_whitelist_auth *@*.bbcmail.co.uk # is not whitelisted
> whitelist_from_dkim *@*.bbcmail.co.uk # is whitelisted
>
> which I presumed meant that both lines were present in your config and
> mail from bbc@e.bbcmail.co.uk was hitting USER_IN_DKIM_WHITELIST, but
> not USER_IN_DEF_DKIM_WL.

Nope. I have been testing this on one single message. I have scanned the
message multiple times, each time changing only one single line in my
config - the whitelist line.

I ran the two whitelist lines in unison only upon the request to see debug
from them being in the same scan. This was the only variation in testing
again.

> I wanted to see the debug for the case where one rule works as
> expected and the other doesn't - and it's important to see this on
> the same mail.
>
> The first two debug entries below show both def and normal whitelist
> hits working properly, but they are confused by a *@bbcmail.co.uk
> whitelisting entry which you say fixes the problem. Below that you
> have a def_whitelist_auth match with *@bbcmail.co.uk. All that you are
> illustrating here is that *@bbcmail.co.uk works as expected, not that
> *@*bbcmail.co.uk fails on anything.

Well the whitelisting failure was the first debug I posted, to clarify.

When using (only):

def_whitelist_auth *@*.bbcmail.co.uk

The debug is:

Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by e.bbcmail.co.uk,
author bbc@e.bbcmail.co.uk, no valid matches
Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk, not in
any dkim whitelist

Therefore the following all successfully whitelist the test mail from
bbc@e.bbcmail.co.uk (each tested individually and not in a combined scan):

whitelist_from_dkim *@*.bbcmail.co.uk
whitelist_from_dkim *@e.bbcmail.co.uk
whitelist_from_dkim bbc@e.bbcmail.co.uk
def_whitelist_auth *@e.bbcmail.co.uk
def_whitelist_auth bbc@e.bbcemail.co.uk

However the following does not whitelist the test mail:

def_whitelist_auth *@*.bbcmail.co.uk




>> Mar 23 15:26:55.945 [27188] dbg: dkim: VALID author domain signature
>> by e.bbcmail.co.uk, MATCHES whitelist_from_dkim
>> (?^i:^.*\@.*\.bbcmail\.co\.uk$)
>> Mar 23 15:26:55.945 [27188] dbg: dkim: author bbc@e.bbcmail.co.uk,
>> WHITELISTED by whitelist_from_dkim/e.bbcmail.co.uk
>>
>> And def_whitelist_auth will match on both the full address:
>>
>> Mar 23 11:21:52.535 [15896] dbg: dkim: VALID author domain signature
>> by e.bbcmail.co.uk, MATCHES def_whitelist_auth
>> ^bbc\@e\.bbcmail\.co\.uk$ Mar 23 11:21:52.535 [15896] dbg: dkim:
>> author bbc@e.bbcmail.co.uk, WHITELISTED by
>> def_whitelist_auth/e.bbcmail.co.uk
>>
>> And also if I only wildcard the user part:
>>
>> Mar 23 11:25:12.688 [16086] dbg: dkim: VALID author domain signature
>> by e.bbcmail.co.uk, MATCHES def_whitelist_auth
>> ^.*\@e\.bbcmail\.co\.uk$ Mar 23 11:25:12.688 [16086] dbg: dkim:
>> author bbc@e.bbcmail.co.uk, WHITELISTED by
>> def_whitelist_auth/e.bbcmail.co.uk
>>
>> It only fails on def_whitelist_auth *@*.bbcmail.co.uk
>>
>> Same message each time, the only variation on the scans are the
>> whitelist settings.
Re: def_whitelist_auth inconsistencies [ In reply to ]
On 23.03.16 16:06, Kevin Golding wrote:
>Well the whitelisting failure was the first debug I posted, to clarify.
>
>When using (only):
>
>def_whitelist_auth *@*.bbcmail.co.uk
>
>The debug is:
>
>Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
>e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
>Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk,
>not in any dkim whitelist
>
>Therefore the following all successfully whitelist the test mail from
>bbc@e.bbcmail.co.uk (each tested individually and not in a combined
>scan):

please put these two lined to your config at once:

>whitelist_from_dkim *@*.bbcmail.co.uk
>def_whitelist_auth *@*.bbcmail.co.uk

and run the mail through spamassassin -D
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
One OS to rule them all, One OS to find them,
One OS to bring them all and into darkness bind them
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 16:33:16 -0000, Matus UHLAR - fantomas
<uhlar@fantomas.sk> wrote:

> On 23.03.16 16:06, Kevin Golding wrote:
>> Well the whitelisting failure was the first debug I posted, to clarify.
>>
>> When using (only):
>>
>> def_whitelist_auth *@*.bbcmail.co.uk
>>
>> The debug is:
>>
>> Mar 23 11:17:44.805 [15610] dbg: dkim: VALID signature by
>> e.bbcmail.co.uk, author bbc@e.bbcmail.co.uk, no valid matches
>> Mar 23 11:17:44.805 [15610] dbg: dkim: author bbc@e.bbcmail.co.uk, not
>> in any dkim whitelist
>>
>> Therefore the following all successfully whitelist the test mail from
>> bbc@e.bbcmail.co.uk (each tested individually and not in a combined
>> scan):
>
> please put these two lined to your config at once:
>
>> whitelist_from_dkim *@*.bbcmail.co.uk
>> def_whitelist_auth *@*.bbcmail.co.uk
>
> and run the mail through spamassassin -D

Oddly, that gets a different result this time:

Mar 23 17:44:24.418 [33341] dbg: dkim: VALID author domain signature by
e.bbcmail.co.uk, MATCHES def_whitelist_auth ^.*\@.*\.bbcmail\.co\.uk$
Mar 23 17:44:24.418 [33341] dbg: dkim: VALID author domain signature by
e.bbcmail.co.uk, MATCHES whitelist_from_dkim
(?^i:^.*\@.*\.bbcmail\.co\.uk$)
Mar 23 17:44:24.418 [33341] dbg: dkim: author bbc@e.bbcmail.co.uk,
WHITELISTED by def_whitelist_auth/e.bbcmail.co.uk,
whitelist_from_dkim/e.bbcmail.co.uk

Which led to more testing and the lone def_whitelist_auth
*@*.bbcmail.co.uk line works too.

I think I may have got caught out by a change of domains at the BBC.
Checking my changelogs I originally had *@*.bbcemail.co.uk - this
logically worked fine for a while, however I noticed it not doing so
lately.

In my testing I added in other rules using the current domain
*.bbcmail.co.uk and they worked, without noticing the original typo. Even
transcribing it for the list I used the new domain instead of the original
rule. So when I started afresh for the latest test it worked fine, and
that's when the error got flagged up.

Cheers chaps! PEBCAK
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016, Kevin Golding wrote:

> Even transcribing it for the list I used the new domain instead of the
> original rule.

I was going to ask about that, but I figured it was just a typo so I
didn't.

--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The difference between ignorance and stupidity is that the stupid
desire to remain ignorant. -- Jim Bacon
-----------------------------------------------------------------------
92 days since the first successful real return to launch site (SpaceX)
Re: def_whitelist_auth inconsistencies [ In reply to ]
On Wed, 23 Mar 2016 17:21:32 -0000, John Hardin <jhardin@impsec.org> wrote:

> On Wed, 23 Mar 2016, Kevin Golding wrote:
>
>> Even transcribing it for the list I used the new domain instead of the
>> original rule.
>
> I was going to ask about that, but I figured it was just a typo so I
> didn't.
>

Never underestimate my ability to make an obvious mistake :-D