Speed up queue injection
in [Postfix]
|
From: Ram on 13 Aug 2010 01:45 We have a requirement to send some research analysis mails as quickly as possible. Everyday after the data is available my app generates the mails in eml format in a directory. These are personalized mails with attachments and have to reach the recipients "instantly" ( in my customers lingo ... Research reports are useless after the market opens ) What is the quickest way of pushing EML files to postfix for delivery. Currently I have a perl script that makes parallel smtp connections on localhost and sends the mails. Should I send the mails on command line. There are currently around 50k mails to be delivered ideally within 5-10 mins. I am only bothered about sending to postfix because delievery from there is already taken care of. Is there a better way , other than sending mails on command line or SMTP. Something like and API to inject into postfix maildrop. Thanks Ram
From: Ralf Hildebrandt on 13 Aug 2010 03:39 * Ram <ram(a)netcore.co.in>: > We have a requirement to send some research analysis mails as quickly as > possible. > > > > Everyday after the data is available my app generates the mails in eml > format in a directory. What is eml format? > Currently I have a perl script that makes parallel smtp connections on > localhost and sends the mails. This sounds good! > Should I send the mails on command line. No, using the postfix sendmail binary is actually slower. > There are currently around 50k mails to be delivered ideally within > 5-10 mins. How fast are you now? 50.000/10min = 5.000/min = 83/s = that's a lot 50.000/50min = 10.000/min = 186/s = that's even more -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de
From: Ram on 13 Aug 2010 05:53 Hi , On Fri, 2010-08-13 at 09:39 +0200, Ralf Hildebrandt wrote: > * Ram <ram(a)netcore.co.in>: > > We have a requirement to send some research analysis mails as quickly as > > possible. > > > > > > > > Everyday after the data is available my app generates the mails in eml > > format in a directory. > > What is eml format? > Mail in plain text format , mime encoded message > > Currently I have a perl script that makes parallel smtp connections on > > localhost and sends the mails. > > This sounds good! > > > Should I send the mails on command line. > > No, using the postfix sendmail binary is actually slower. > > > There are currently around 50k mails to be delivered ideally within > > 5-10 mins. > > How fast are you now? > 50.000/10min = 5.000/min = 83/s = that's a lot > 50.000/50min = 10.000/min = 186/s = that's even more > Currenlty I get 40/s - 45/s But I want it to be atleast 100/s Delivery is not at all an issue , because postfix gives it to further relay boxes which are under our control again. This is a 8GB Ram Centos 5.4 server with SAS discs Do I need to increase the hardware Thanks Ram
From: Ralf Hildebrandt on 13 Aug 2010 07:30 * Ram <ram(a)netcore.co.in>: > Mail in plain text format , mime encoded message OK! > Currenlty I get 40/s - 45/s That sounds normal. Any filtering (in these cases you should inject in a way that bypasses and filters) > But I want it to be atleast 100/s Two machineS? relay boxes > Delivery is not at all an issue , because postfix gives it to further > relay boxes which are under our control again. Why not inject to the further relay boxes? > Do I need to increase the hardware It could be :) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de
From: Wietse Venema on 13 Aug 2010 08:23
Ralf Hildebrandt: > * Ram <ram(a)netcore.co.in>: > > > Mail in plain text format , mime encoded message > > OK! > > > Currenlty I get 40/s - 45/s > That sounds normal. Any filtering (in these cases you should inject in > a way that bypasses and filters) > > But I want it to be atleast 100/s > > Two machineS? > relay boxes > > Delivery is not at all an issue , because postfix gives it to further > > relay boxes which are under our control again. > > Why not inject to the further relay boxes? > > > Do I need to increase the hardware > > It could be :) Other options: increase input concurrency, or play with in_flow_delay. Note that increasing your input rates will cause output rates to drop. It's all about competing for disk access. Wietse |