Mailing List Archive

Exim cluster in front of Amazon SES
We're looking to use Amazon's SES service in our new AWS environment, but
we're very wary of getting blocked by them for going over the quota or
dealing with customers configuring from addresses that haven't been
verified in SES with the DKIM keys appropriately yet. Part of that involves
updating our product to disallow new domains to be used without prior
approval, but that's not going to be ready right away. So our thought was
to set up a simple exim cluster behind an ELB that would:

1.) Accept mail from our various webservers on the local network using
security groups
2.) Verify that the domain is a member of a domain whitelist and if so,
send it out SES as a smarthost
3.) Rate limit how fast we forward over to SES to avoid sending limits
being breached

I've already gone through the process of configuring SES as a smarthost via
a Chef cookbook, involving configuring the authentication and such; that
all works without issue.

For rate limiting, I'm having some difficulty; I see how I can limit
inbound mail through the various ACLs, but I'm not seeing how to control
things granularly once they get into the queue. I see that there are some
options I can set like connection_max_messages at the transport
layer, remote_max_parallel, queue_run_max, and serialize_hosts -- all
options that I can use to degrade performance to the smarthost, but these
aren't really rate limiting per se, they're more reducing the maximum
capacity of the transport/queue system.

For the domain whitelisting, I'm not seeing any way to do that kind of
verification after the message is accepted. We may need to skip that and
just monitor the frozen queue items to see what mails are queued in exim
but not accepted in SES via some form of script that reports on domains
that are in the queue but not yet verified in SES. This is more of a
stop-gap solution until the product update is made so I'm less concerned on
this front. We could also set up more simplistic exim configs on the Linux
webservers and have them use the cluster as a smarthost, and then issue
temporary rejections for mail not from domains in the whitelist at the
cluster so that they would queue up on the webservers. However, I'm not
really keen on that kind of solution.

Any suggestions on these fronts would be greatly appreciated. We're doing a
phased rollout approach to AWS, so ideally the product will be updated
before too many other services are migrated over to AWS, but we're
definitely concerned about getting blocked and affecting multiple
customers/services in a given region.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: Exim cluster in front of Amazon SES [ In reply to ]
On 04/05/18 19:58, Morgan Blackthorne via Exim-users wrote:
> 1.) Accept mail from our various webservers on the local network using
> security groups
> 2.) Verify that the domain is a member of a domain whitelist and if so,
> send it out SES as a smarthost
> 3.) Rate limit how fast we forward over to SES to avoid sending limits
> being breached

Outbound rate-limiting is hard. See here for some ideas:

https://github.com/Exim/exim/wiki/RatelimitOutbound

> For the domain whitelisting, I'm not seeing any way to do that kind of
> verification after the message is accepted. We may need to skip that and

a) Why accept them in the first place?
b) divert them to an alternate named-queue, on which you are not
operating a queue-runner, pending a manual selective run of that
queue?

--
Cheers,
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/