Mailing List Archive

lmtpd performance
Hi,

Today we had a few issues which caused a back log of mail (around 16000 emails to queue up on postfix servers). When the issue seemed to be finally sorted lmtpd struggled to process the queue inserting around 1 or 2 emails per second. Am I being impatient and this is normal? I seem to recall dbmail 2.2 being able to churn through emails and insert them into mysql a lot quicker.

Just to confirm dbmail pop3d and imapd are all running (and busy) updating, reading etc suitably quickly. We run the system on ZFS with SSD write and SSD read caches and mysql is tuned. There are no slow queries and show processlist never has anything in it. It seems like lmtpd is just handling only 1 at a time i.e. not multi threaded?

Jun 13 15:28:44 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [19] mailbox [82] message [15047635] size [25880] is inserted
Jun 13 15:28:45 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [4651] mailbox [20107] message [15047637] size [2751] is inserted
Jun 13 15:28:46 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [4870] mailbox [21135] message [15047641] size [22597] is inserted
Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [2353] mailbox [10460] message [15047643] size [90509] is inserted
Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [2289] mailbox [10227] message [15047645] size [10100] is inserted
Jun 13 15:28:48 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [2847] mailbox [12396] message [15047647] size [23445] is inserted
Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [3194] mailbox [13819] message [15047649] size [81818] is inserted
Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [4872] mailbox [21140] message [15047651] size [61540] is inserted
Jun 13 15:28:50 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [824] mailbox [3504] message [15047653] size [14261] is inserted
Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [4559] mailbox [19593] message [15047655] size [31050] is inserted
Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [3520] mailbox [15205] message [15047657] size [30910] is inserted
Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [6505] mailbox [28584] message [15047659] size [7388] is inserted
Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [6137] mailbox [26902] message [15047661] size [123777] is inserted
Jun 13 15:28:56 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [1158] mailbox [4976] message [15047663] size [142518] is inserted
Jun 13 15:28:57 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] sort_deliver_to_mailbox(+2260): useridnr [3098] mailbox [13445] message [15047665] size [2357] is inserted

Any thoughts welcome, maybe I'm missing something it just seems a bit slow that's all and I think I'll be here a while getting 16k+ emails into the db for customers.

James G
Re: lmtpd performance [ In reply to ]
dbmail-lmtpd being single threaded, 1 or 2 per second per process is
quite normal.

short of making lmtpd multi-threaded:

If you need more throughput, you could spin up more lmtpd process on
different ports and have postfix load-balance over them using something
like haproxy.

Never done that myself, but it should work.



On 13-06-14 16:30, James Greig wrote:
> Hi,
>
>
>
> Today we had a few issues which caused a back log of mail (around 16000
> emails to queue up on postfix servers). When the issue seemed to be
> finally sorted lmtpd struggled to process the queue inserting around 1
> or 2 emails per second. Am I being impatient and this is normal? I
> seem to recall dbmail 2.2 being able to churn through emails and insert
> them into mysql a lot quicker.
>
>
>
> Just to confirm dbmail pop3d and imapd are all running (and busy)
> updating, reading etc suitably quickly. We run the system on ZFS with
> SSD write and SSD read caches and mysql is tuned. There are no slow
> queries and show processlist never has anything in it. It seems like
> lmtpd is just handling only 1 at a time i.e. not multi threaded?
>
>
>
> Jun 13 15:28:44 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [19] mailbox [82] message
> [15047635] size [25880] is inserted
>
> Jun 13 15:28:45 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4651] mailbox [20107] message
> [15047637] size [2751] is inserted
>
> Jun 13 15:28:46 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4870] mailbox [21135] message
> [15047641] size [22597] is inserted
>
> Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2353] mailbox [10460] message
> [15047643] size [90509] is inserted
>
> Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2289] mailbox [10227] message
> [15047645] size [10100] is inserted
>
> Jun 13 15:28:48 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2847] mailbox [12396] message
> [15047647] size [23445] is inserted
>
> Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3194] mailbox [13819] message
> [15047649] size [81818] is inserted
>
> Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4872] mailbox [21140] message
> [15047651] size [61540] is inserted
>
> Jun 13 15:28:50 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [824] mailbox [3504] message
> [15047653] size [14261] is inserted
>
> Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4559] mailbox [19593] message
> [15047655] size [31050] is inserted
>
> Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3520] mailbox [15205] message
> [15047657] size [30910] is inserted
>
> Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [6505] mailbox [28584] message
> [15047659] size [7388] is inserted
>
> Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [6137] mailbox [26902] message
> [15047661] size [123777] is inserted
>
> Jun 13 15:28:56 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [1158] mailbox [4976] message
> [15047663] size [142518] is inserted
>
> Jun 13 15:28:57 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3098] mailbox [13445] message
> [15047665] size [2357] is inserted
>
>
>
> Any thoughts welcome, maybe I’m missing something it just seems a bit
> slow that’s all and I think I’ll be here a while getting 16k+ emails
> into the db for customers.
>
>
>
> James G
>
>
>
> _______________________________________________
> DBmail mailing list
> DBmail@dbmail.org
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, github, linkedin
www.nfg.nl/info@nfg.nl/+31.85.877.99.97
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: lmtpd performance [ In reply to ]
Hi Paul,

> dbmail-lmtpd being single threaded, 1 or 2 per second per process is
> quite normal.
>
> short of making lmtpd multi-threaded:
>
> If you need more throughput, you could spin up more lmtpd process on
> different ports and have postfix load-balance over them using something
> like haproxy.
>
> Never done that myself, but it should work.


This is quite interesting.

To achieve this can I call multiple dbmail-lmtpd processes specifying
different config files to make them run in different ports as you said?

_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: lmtpd performance [ In reply to ]
Yes Jorge, that would be the idea. Separate configs.

On 15 juni 2014 19:26:32 CEST, Jorge Bastos <mysql.jorge@decimal.pt> wrote:
>Hi Paul,
>
>> dbmail-lmtpd being single threaded, 1 or 2 per second per process is
>> quite normal.
>>
>> short of making lmtpd multi-threaded:
>>
>> If you need more throughput, you could spin up more lmtpd process on
>> different ports and have postfix load-balance over them using
>something
>> like haproxy.
>>
>> Never done that myself, but it should work.
>
>
>This is quite interesting.
>
>To achieve this can I call multiple dbmail-lmtpd processes specifying
>different config files to make them run in different ports as you said?
>
>_______________________________________________
>DBmail mailing list
>DBmail@dbmail.org
>http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

--
Verzonden van mijn Android telefoon met K-9 Mail.
Re: lmtpd performance [ In reply to ]
Are there any plans in the future to make it multi-threaded at all? I presume in a way postfix piping the mail to dbmail-smtp would actually be quicker (though less preferred).

I did consider load balancing lmtp just for the sake of clearing down the queue as at times it almost seemed like the amount of mail coming in out weighed the speed at which it was inserting particularly with a large backlog that we had.

James Greig

-----Original Message-----
From: dbmail-bounces@dbmail.org [mailto:dbmail-bounces@dbmail.org] On Behalf Of Paul J Stevens
Sent: 15 June 2014 15:06
To: DBMail mailinglist
Subject: Re: [Dbmail] lmtpd performance


dbmail-lmtpd being single threaded, 1 or 2 per second per process is quite normal.

short of making lmtpd multi-threaded:

If you need more throughput, you could spin up more lmtpd process on different ports and have postfix load-balance over them using something like haproxy.

Never done that myself, but it should work.



On 13-06-14 16:30, James Greig wrote:
> Hi,
>
>
>
> Today we had a few issues which caused a back log of mail (around
> 16000 emails to queue up on postfix servers). When the issue seemed
> to be finally sorted lmtpd struggled to process the queue inserting
> around 1 or 2 emails per second. Am I being impatient and this is
> normal? I seem to recall dbmail 2.2 being able to churn through
> emails and insert them into mysql a lot quicker.
>
>
>
> Just to confirm dbmail pop3d and imapd are all running (and busy)
> updating, reading etc suitably quickly. We run the system on ZFS with
> SSD write and SSD read caches and mysql is tuned. There are no slow
> queries and show processlist never has anything in it. It seems like
> lmtpd is just handling only 1 at a time i.e. not multi threaded?
>
>
>
> Jun 13 15:28:44 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [19] mailbox [82] message
> [15047635] size [25880] is inserted
>
> Jun 13 15:28:45 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4651] mailbox [20107]
> message [15047637] size [2751] is inserted
>
> Jun 13 15:28:46 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4870] mailbox [21135]
> message [15047641] size [22597] is inserted
>
> Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2353] mailbox [10460]
> message [15047643] size [90509] is inserted
>
> Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2289] mailbox [10227]
> message [15047645] size [10100] is inserted
>
> Jun 13 15:28:48 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [2847] mailbox [12396]
> message [15047647] size [23445] is inserted
>
> Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3194] mailbox [13819]
> message [15047649] size [81818] is inserted
>
> Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4872] mailbox [21140]
> message [15047651] size [61540] is inserted
>
> Jun 13 15:28:50 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [824] mailbox [3504] message
> [15047653] size [14261] is inserted
>
> Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [4559] mailbox [19593]
> message [15047655] size [31050] is inserted
>
> Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3520] mailbox [15205]
> message [15047657] size [30910] is inserted
>
> Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [6505] mailbox [28584]
> message [15047659] size [7388] is inserted
>
> Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [6137] mailbox [26902]
> message [15047661] size [123777] is inserted
>
> Jun 13 15:28:56 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [1158] mailbox [4976] message
> [15047663] size [142518] is inserted
>
> Jun 13 15:28:57 mailhost-4a dbmail/lmtpd[4814]: Notice:[message]
> sort_deliver_to_mailbox(+2260): useridnr [3098] mailbox [13445]
> message [15047665] size [2357] is inserted
>
>
>
> Any thoughts welcome, maybe I'm missing something it just seems a bit
> slow that's all and I think I'll be here a while getting 16k+ emails
> into the db for customers.
>
>
>
> James G
>
>
>
> _______________________________________________
> DBmail mailing list
> DBmail@dbmail.org
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, github, linkedin
www.nfg.nl/info@nfg.nl/+31.85.877.99.97
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: lmtpd performance [ In reply to ]
Le dimanche 15 juin 2014 à 16:06 +0200, Paul J Stevens a écrit :
> dbmail-lmtpd being single threaded, 1 or 2 per second per process is
> quite normal.

If DBmail relied on schemas, it would make sense to open a transaction
for each delivery. Each transaction is handled by a different process,
which would scale both at connection level and database level.

> short of making lmtpd multi-threaded:
>
> If you need more throughput, you could spin up more lmtpd process on
> different ports and have postfix load-balance over them using
> something
> like haproxy.

A modern database like PostgreSQL can handle 10.000 queries per second.
Especially when using SSDs. So why should we need multiple hosts and a
complex scenario?

Proxies are never a solution for bad database design and poor security.

The first thing to do is to make each account separate in database using
schemas in PostgreSQL and table prefix in MySQL. THEN on only THEN
DBMail should scale easily, because each database connection will be
handled by a separate process.

Kind regards,
Kellogs

_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: lmtpd performance [ In reply to ]
Am 17.06.2014 15:03, schrieb Jean-Michel Pouré - GOOZE:
> The first thing to do is to make each account separate in database using
> schemas in PostgreSQL and table prefix in MySQL.

oh my god...

single instance storage bye bye

> THEN on only THEN DBMail should scale easily, because each database
> connection will be handled by a separate process

bullshit - dbmail-2.x had as example for IMAP a single
process with it's own db-connection for each client
connection and in case of IMAP idle you have one per
user *and* folder

that may do anything but *not* scale
Re: lmtpd performance [ In reply to ]
On 17-06-14 15:03, Jean-Michel Pouré - GOOZE wrote:

> The first thing to do is to make each account separate in database using
> schemas in PostgreSQL and table prefix in MySQL. THEN on only THEN
> DBMail should scale easily, because each database connection will be
> handled by a separate process.

Not being multi-threaded with a connection pool was what killed
performance on postgresql and oracle with dbmail-2.

One process per database connection is not going to happen (again).


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, github, linkedin
www.nfg.nl/info@nfg.nl/+31.85.877.99.97
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail