Mailing List Archive

NDN, Mailer-Daemon, DSN - EXIM
Hello, we have an issue with Non-Delivery Notification NDN in Exim 4.90_1
There are 2 mail servers Exim and one as Email Spam Filtering - Cisco Ironport (ESA)

The Primary Mail server:
primary_hostname = main1.company.com, qualify_domain = company.com

The Internal server for mass mailing:
primary_hostname = corp17.company.com and the qualify_domain = company.com

As MX server for domain company.com acts Cisco ESA so the all correspondence goes thru for a spam filtration.
SPF, DKIM, and DMARK records configured correctly and everything works as well expect NDN Notification

For SPF used rule for example:
v=spf1 a mx ip4:1.1.1.1 ip4:2.2.2.2 ip4:3.3.3.3 -all


But when someone try to sending an email through Internal server corp17.company.com to non existing email address for example wwwwii@gmail.com
the corp17.company.com forms a NDN mail to user from company.com domain and trying to sent it through MX server - Cisco ESA. As there were SPF
configured for domain company.com Cisco ESA check field and report that

Received-SPF: None (esa.company.com: no sender authenticity information available from domain of
postmaster@corp17.company.com) identity=helo; client-ip=1.1.1.1; receiver=esa.company.com;
envelope-from=""; x-sender="postmaster@corp17.company.com"; x-conformance=spf_only


Off course there is no SPF records for domain: corp17.company.com - its only primary_hostname of the server.

Why "Internal server" use primary_hostname instead qualify_domain ?
I'll trying to set dsn_from = Mail Delivery System <Mailer-Daemon@$qualify_domain> but receive the same message above.



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On 10 Apr 2018, at 16:51, Nazarevych Ol via Exim-users <exim-users@exim.org> wrote:
> Why "Internal server" use primary_hostname instead qualify_domain ?

You most likely need to look at your configuration to find that. You don’t say on what operating system you’re running, nor whether you’ve already got dsn_from set somewhere.

By default, exim does not use postmaster@$primary_hostname, so you’ve almost certainly got either a global setting or something in a router or transport (or a rewrite) that’s doing it for you.

Graeme
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
Hello,

I've read through your question a few times and have a feeling I know what
might be wrong. If I'm right then it's not a problem with Exim, but that
you're missing one or more SPF records.

I think you're saying that:

1. Someone sends a message to the non-existent email address <
wwwwii@gmail.com>
2. This message goes out via the mail service running on
corp17.company.com
3. The server finds the recipient email address is not valid so creates
a Non-Delivery Notification (NDN) to send back to the person who wrote the
original email.
4. This NDN is then either rejected, or received with a warning, by the
Cisco ESA because SPF failed as shown in the "Received-SPF:" header you
quoted.

Is that correct?

If it is, then check you have got an SPF record set up for the name that
the mail server running on corp17.company.com.

*Explanation:*
When you send a normal message the sender address in the SMTP envelope (the
"MAIL FROM" address) is usually that of the person sending the message.
When the receiving mail server performs an SPF check it gets the domain
name of this email address and looks up its SPF record in the DNS, then
checks the IP address of the transmitting server against that record.

However a Non Delivery Notification is different; its sender address in the
SMTP envelope is the special empty address "<>". As you can see, being
empty it has no domain name. So a mail server receiving an NDN can't get a
domain name form there to look up an SPF record.

Instead it uses the string the server used to identify itself in the
HELO/EHLO command it issued when it connected and uses that as the domain
name to look up an SPF record for.

So if your corp17.company.com mail server:

- generates an NDN (ie, a message with an empty MAIL FROM address in the
SMTP envelope), and
- connects to the mail service on a system that checks SPF records, and
- identifies itself in the HELO/EHLO string as "corp17.company.com", then
- the receiving mail system will look for a TXT record in the DNS named "
corp17.company.com" to get the SPF record to check against.

I made exactly the same mistake when I first set up SPF here, not realising
I had to set up an SPF record in the DNS for each of our outgoing mail
servers. (Or, more accurately, the domain name they identify themselves as
in their HELO/EHLO greeting: which happens to also be their actual host
name.)

So here I have:

- an SPF record for our "york.ac.uk" domain so that normal emails have
their MAIL FROM address checked against this record, and
- an SPF record for each mail server that might generate an NDN so that
NDNs (ie, messages with "<>" as the MAIL FROM address) are authenticated
with SPF

For example:

- york.ac.uk. IN TXT "v=spf1 ..."
- mailgw0.york.ac.uk. IN TXT "v=spf1 a -all"
- mailgw1.york.ac.uk. IN TXT "v=spf1 a -all"
- and so on

The individual SPF records for each mail server only need to identify
themselves (by using the "A" mechanism) and no others (by using the "-all"
mechanism).

To confirm this you can read the FAQ/Common Mistakes page at the OpenSPF
web site, in particular the question entitled *Publish SPF records for HELO
names used by your mail servers*:

http://www.openspf.org/FAQ/Common_mistakes#helo


You can also find the recommendations in the RFC for SPF:

- For the HELO identity: https://tools.ietf.org/html/rfc7208#section-2.3
- For bounces (NDNs): https://tools.ietf.org/html/rfc7208#section-10.1.3

*The First Step You Should Do*
The very first thing you should do is check your Exim logs to make sure
what I've written applies to you. That is, that the Non-Delivery
Notification is going out with a MAIL FROM address of "<>" in its SMTP
envelope. If your Exim logs show it it really does have "
postmaster@corp17.company.com" in the MAIL FROM address instead then:

- the solution I describe above doesn't apply to you, and
- your "NDN" isn't actually an NDN! (Those should have an empty "<>"
address as the MAIL FROM in the SMTP envelope.)

Cheers,
Mike B-)

On 10 April 2018 at 16:51, Nazarevych Ol via Exim-users <exim-users@exim.org
> wrote:

> Hello, we have an issue with Non-Delivery Notification NDN in Exim 4.90_1
> There are 2 mail servers Exim and one as Email Spam Filtering - Cisco
> Ironport (ESA)
>
> The Primary Mail server:
> primary_hostname = main1.company.com, qualify_domain = company.com
>
> The Internal server for mass mailing:
> primary_hostname = corp17.company.com and the qualify_domain = company.com
>
> As MX server for domain company.com acts Cisco ESA so the all
> correspondence goes thru for a spam filtration.
> SPF, DKIM, and DMARK records configured correctly and everything works as
> well expect NDN Notification
>
> For SPF used rule for example:
> v=spf1 a mx ip4:1.1.1.1 ip4:2.2.2.2 ip4:3.3.3.3 -all
>
>
> But when someone try to sending an email through Internal server
> corp17.company.com to non existing email address for example
> wwwwii@gmail.com
> the corp17.company.com forms a NDN mail to user from company.com domain
> and trying to sent it through MX server - Cisco ESA. As there were SPF
> configured for domain company.com Cisco ESA check field and report that
>
> Received-SPF: None (esa.company.com: no sender authenticity information
> available from domain of
> postmaster@corp17.company.com) identity=helo; client-ip=1.1.1.1;
> receiver=esa.company.com;
> envelope-from=""; x-sender="postmaster@corp17.company.com";
> x-conformance=spf_only
>
>
> Off course there is no SPF records for domain: corp17.company.com - its
> only primary_hostname of the server.
>
> Why "Internal server" use primary_hostname instead qualify_domain ?
> I'll trying to set dsn_from = Mail Delivery System <Mailer-Daemon@
> $qualify_domain> but receive the same message above.
>
>
>
> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> ## Exim details at http://www.exim.org/
> ## Please use the Wiki with this list - http://wiki.exim.org/
>



--
Systems Administrator & Change Manager
IT Services, University of York, Heslington, York YO10 5DD, UK
Tel: +44-(0)1904-323811

Web: www.york.ac.uk/it-services
Disclaimer: www.york.ac.uk/docs/disclaimer/email.htm
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On 2018-04-11 13:28, Graeme Fowler wrote:

> On 10 Apr 2018, at 16:51, Nazarevych Ol via Exim-users <exim-users@???> wrote:

> > Why "Internal server" use primary_hostname instead qualify_domain ?

>

> You most likely need to look at your configuration to find that. You don?t say on what operating system you?re running, nor whether you?ve already got dsn_from set somewhere.

>

> By default, exim does not use postmaster@$primary_hostname, so you?ve almost certainly got either a global setting or something in a router or transport (or a rewrite) that?s doing it for you.

>

> Graeme

>

Hello, thanks fot the answer. This is a clear CentOS Linux release 7.4.1708 (Core) install with Exim version 4.90_1 #2 built 16-Feb-2018 16:47:02
The field dsn_from was set to:

dsn_from = Mail Delivery System <Mailer-Daemon@$qualify_domain>

But this does not does not affect to SPF check on another side on Cisco ESA, and Mail delivery reports looks like:

From: Mailer-Daemon@company.com - there were dsn_from applied, if we change dsn_from for something accurate as Mailer-Daemon@$exeactly.com - we can see that field changed in a NDN letter.

But this is an declarative fields and real headers looks like:

Received-SPF: None (esa.company.com: no sender authenticity
information available from domain of
postmaster@corp17.company.com) identity=mailfrom;
client-ip=1.1.1.1; receiver=esa.company.com;
envelope-from=""; x-sender="postmaster@corp17.company.com";
x-conformance=spf_only
Received-SPF: None (esa.company.com: no sender authenticity
information available from domain of
postmaster@corp17.company.com) identity=helo;
client-ip=1.1.1.1; receiver=esa.company.com;
envelope-from=""; x-sender="postmaster@corp17.company.com";
x-conformance=spf_only

Authentication-Results: esa.company.com; spf=None smtp.mailfrom=postmaster@corp17.company.com; spf=None smtp.helo=postmaster@corp17.company.com; dkim=pass (signature verified) header.i=@company.com; spf=None smtp.helo=postmaster@corp17.company.com; dmarc=pass (p=reject dis=none) d=company.com
Received: from corp17.company.com ([1.1.1.1])
by esa.company.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Apr 2018 18:06:39 +0300
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;

Received: from exim by corp17.company.com with local (Exim 4.90_1)
id 1f5upx-0002G0-08
for it@company.com; Tue, 10 Apr 2018 18:05:33 +0300
X-Failed-Recipients: wwwwiiit@gmail.com
Auto-Submitted: auto-replied
From: Mail Delivery System <Mailer-Daemon@company.com> -------------- this is correct, but fail SPF check above

To: itp@company.com
Subject: Mail delivery failed: returning message to sender
Date: Tue, 10 Apr 2018 18:05:33 +0300
Content-type: text/plain; charset=us-ascii

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
To Author: Mike Brudenell
Thanks for the reply. Yes you are fully correct how it looks like. I'm mean the stages.

Ok, I'm I completely agree with empty sender <> and read about it, so there were no problem to add additional spf record for the domain name.
But in our case - we've use around 65 domains and 3 mail server to do mass mail for our clients whos registered on our sites to made an news letters, or some transactional emails like orders on a site.

This is about 195 new spf records that we should to create and always maintain.
Become acquainted with documents provided, looks like there were only one correct way - to create such records.
Hoped maybe in a future Exim lets us to use some variable in a transport also for NDN reports to replace <> with desired value.

PS. Sorry for incorect answer in a post - I could not understand how to reply to a message on this lists.


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
Hello,

Be careful: it might not be 65*3 = 195 new SPF records that you need. It
might only be three! Remember that it is whatever your Exim (in this case I
think the Exim running on the server known as corp17.company.com) puts into
its HELO/EHLO command when it connects to another server.

By default this is the primary host name of your server, as set with
primary_hostname (or if you don't set it with this then the DNS name of the
server).

Looking back at your original question you say:

The Internal server for mass mailing:
primary_hostname = corp17.company.com and the qualify_domain = company.com


That means you're setting the primary hostname to "corp17.company.com" and
it's this that Exim always use in its HELO/EHLO when it connects to another
mail server. (But see the note below!)

That means you only need a new SPF record for the *corp17.company.com
<http://corp17.company.com>* domain name to correct that problem for that
entire mail server. You *don't* need one for every domain it happens to
send out emails from.

*Note:* The exception is if you use a *helo_data* command within your Exim
transport to set the HELO/EHLO string for each connection to, say, reflect
the domain name of the sender of the outgoing message instead of the
default action of always using the primary hostname. If you're changing the
value of the domain name in the HELO/EHLO greeting in this way then yes,
you'll need an SPF record for each domain name. (But then it would only be
65, not 3*65)

I suspect you'll only need 3 new ones in all, as most people don't use
helo_data to alter the HELO/EHLO greeting.

Cheers,
Mike B-)

--
Systems Administrator & Change Manager
IT Services, University of York, Heslington, York YO10 5DD, UK
Tel: +44-(0)1904-323811

Web: www.york.ac.uk/it-services
Disclaimer: www.york.ac.uk/docs/disclaimer/email.htm
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On 11/04/18 17:27, Mike Brudenell via Exim-users wrote:
> a *helo_data* command within your Exim
> transport to set the HELO/EHLO string for each connection to, say, reflect
> the domain name of the sender of the outgoing message instead of the
> default action of always using the primary hostname.

That would be incorrect per standards. The HELO name should identify
the sending system.
--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On 11 April 2018 at 19:43, Jeremy Harris via Exim-users <exim-users@exim.org
> wrote:

> That would be incorrect per standards. The HELO name should identify
> the sending system.
>

Umm… Agreed, but what if this mysterious and obfuscated organisation has
multiple A records under different domain names pointing at the system Exim
is running on? Wouldn't it then be valid to use any of those in the
HELO/EHLO command? E.g.,

- a.example.com is an A record pointing at ww.xx.yy.zz
- b.example.net is an A record pointing at ww.xx.yy.zz

Wouldn't it then be permissible to use a.example.com in the HELO/EHLO when
sending from an "@a.example.com" address, but b.example.net when sending
from an "@b.example.net" address?

RFC 5321 section 4.1.1.1 states: "The argument clause contains the
fully-qualified domain name of the SMTP client, if one is available." which
both of the above examples would fit. (The RFC doesn't explicitly state it
has to be the rDNS name of the server, although one could argue the
following sentence — "In situations in which the SMTP client system does
not have a meaningful domain name (e.g., when its address is dynamically
allocated and no reverse mapping record is available), the client SHOULD
send an address literal" — might be taken to imply it.)

If it is the case that the primary host name of the server MUST be used,
I'm struggling to think of a use-case when Exim's helo_data could be used
validly; I'd (possibly incorrectly?) assume it was for the above type of
scenario. Is it instead, say, only for use when sending out from Exim
running on a server with multiple interfaces?

Cheers,
Mike B-)

--
Systems Administrator & Change Manager
IT Services, University of York, Heslington, York YO10 5DD, UK
Tel: +44-(0)1904-323811

Web: www.york.ac.uk/it-services
Disclaimer: www.york.ac.uk/docs/disclaimer/email.htm
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On April 11, 2018 9:43:10 PM GMT+03:00, Jeremy Harris via Exim-users <exim-users@exim.org> wrote:
>On 11/04/18 17:27, Mike Brudenell via Exim-users wrote:
>> a *helo_data* command within your Exim
>> transport to set the HELO/EHLO string for each connection to, say,
>reflect
>> the domain name of the sender of the outgoing message instead of the
>> default action of always using the primary hostname.
>
>That would be incorrect per standards. The HELO name should identify
>the sending system.

Then what about hosting servers with multiple interfaces? I mean, if i dedicate particular IP to a client, then he probably did not want, that other clients affect its IP reputation. Thus, a simple solution would be to specify 'interface' and matching to used IP's rdns 'helo_data' .

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: NDN, Mailer-Daemon, DSN - EXIM [ In reply to ]
On 12/04/18 09:42, Mike Brudenell via Exim-users wrote:
> On 11 April 2018 at 19:43, Jeremy Harris via Exim-users <exim-users@exim.org
>> wrote:
>
>> That would be incorrect per standards. The HELO name should identify
>> the sending system.
>>
>
> Umm… Agreed, but what if this mysterious and obfuscated organisation has
> multiple A records under different domain names pointing at the system Exim
> is running on? Wouldn't it then be valid to use any of those in the
> HELO/EHLO command? E.g.,
>
> - a.example.com is an A record pointing at ww.xx.yy.zz
> - b.example.net is an A record pointing at ww.xx.yy.zz
>
> Wouldn't it then be permissible to use a.example.com in the HELO/EHLO when
> sending from an "@a.example.com" address, but b.example.net when sending
> from an "@b.example.net" address?

Yes. Because both cases identify the sending system.
--
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/