Prev: O Poder do DOIS: Uma rota simples ao Sucesso... Responda para você mesmo essas perguntas: Você pode adquirir um Pacote Standard por mês por US$20 dólares? Você pode afiliar todos os meses DUAS pessoas que façam o mesmo que VOCÊ? BEM... Se você foc
Next: Sendmail queue runner query - with deep queues starting from scratch...
From: JonB on 27 Jan 2010 03:56 On Jan 26, 6:45 pm, Andrzej Adam Filip <a...(a)onet.eu> wrote: > Have you considered using both "queue run by daemon" without hoststatus > and "queue runs from cron" with hoststatus? Hmmm, no - I hadn't... Interesting, though I'm not too sure what that will get me... As an example of the problem - a server today has 34k messages in it's queue (which isn't *that* many) and has been running uninterrupted several hours. It's running (for better or for worse): sendmail -q30s As a background process to launch queue runners. Number of queue runners running at the moment? 21. Each queue runner is taking just shy of an hour to go through messages it finds in the queue. 'MaxRunnersPerQueue' on this sever is set to 230. If they take nearly an hour to finish, and it's launching one every 30 seconds - how come there are only 21 running? If I manually launch at 5 second intervals a whole bunch (say, 50 queue runners) they all go and find mail to try [and a whole bunch gets delivered]. It's almost like 'sendmail -q30s' is ultra conservative about launching runners... The machine isn't heavily loaded (LA = 0.04 at the moment) - there is no disk bottleneck there is plenty of memory - but it just isn't launching as many queue runners as you would expect (or as are needed). I might try removing 'sendmail -q30s' on this server today and instead run up say 80 'sendmail -qp's - though in the documentation I've seen it says: "With -qp sendmail at the start of every queue run reads all the qf files, sorts them and uses it to control the order in which the queue is run, it then **forks multiple processes** to deliver the mail it has sorted and sleeps the queue interval before awakening again. This ensures no other queue runners are started while it's reading the queue". No where can I find out how many 'multiple processes' is. Is is up to the 'MaxRunnersPerQueue' limit? - Is it a standard 'One queue runner per queue group'? Guess I'll have to go and find out :) -Jon
From: JonB on 27 Jan 2010 04:08 On Jan 27, 8:56 am, JonB <jfre...(a)googlemail.com> wrote: > "With -qp sendmail at the start of every queue run reads all the qf > files, sorts them and uses it to control the order in which the queue > is run, it then **forks multiple processes** to deliver the mail it > has sorted and sleeps the queue interval before awakening again. This > ensures no other queue runners are started while it's reading the > queue". > > No where can I find out how many 'multiple processes' is. Is is up to > the 'MaxRunnersPerQueue' limit? - Is it a standard 'One queue runner > per queue group'? > > Guess I'll have to go and find out :) To answer my own question - it appears to fork up as many queue runners as is allowed in the .cf file as 'MaxRunnersPerQueue' - the system originally had 230 odd running. However after a promising start this has fallen to 90 runners now. I just hope '-qp' doesn't hold off on doing another 'read the queue and fork' run until *every* queue runner it started has finished... I have a horrible feeling it might. -Jon
From: David F. Skoll on 27 Jan 2010 20:00 JonB wrote: > The first queue runner runs, reads the entire queue (which afaik - is > what they do even if in 'random' queue sort order - I think even > 'filename' order will still read the entire queue 'filename' calls readdir() a bunch of times, but I don't believe it actually opens and reads each qf file. I think the parameter you want is confMAX_QUEUE_RUN_SIZE aka MaxQueueRunSize. If you set that to 1000, then the queue runners stop reading the queue once they hit 1000 entries and start processing it. If you run lots of queue runners, your queue will be processed in manageable chunks with a relatively low start-up cost for each run. Usual caveats about potential out-of-order delivery apply. Regards, David.
From: JonB on 28 Jan 2010 05:31 On Jan 28, 1:00 am, "David F. Skoll" <d...(a)roaringpenguin.com> wrote: > JonB wrote: > > The first queue runner runs, reads the entire queue (which afaik - is > > what they do even if in 'random' queue sort order - I think even > > 'filename' order will still read the entire queue > > 'filename' calls readdir() a bunch of times, but I don't believe it > actually opens and reads each qf file. > > I think the parameter you want is confMAX_QUEUE_RUN_SIZE aka > MaxQueueRunSize. If you set that to 1000, then the queue runners stop > reading the queue once they hit 1000 entries and start processing it. If > you run lots of queue runners, your queue will be processed in manageable > chunks with a relatively low start-up cost for each run. I don't think it's the actual startup cost that's an issue - the biggest problem is where people have 3 MX's listed, and each waits the full 10 minutes timeout for "end of data" before that forked delivery process ends... Without sufficient deliveries going on - you run the risk of having other 'candidates' languishing in the queue. So far, I've found: - Limiting the max queue runners per queue to 16 - Running 16 * 'sendmail -qp' (staggered) Gives the kind of kind of 'out the gate' performance we need when we for when we're starting from cold - and also seems to keep considerably (>200) deliverers up and running consistantly (as opposed to 40 odd previously). Sure - some may find they have no work to do (especially as queues empty) but I rather that than having older entries not attended to in time. Longer term we may look to moving hard to deliver email to seperate queues - I think the last time we looked at that (few years ago) - there were issues because our queue directories are spread over spindle sets (something to do with hard links, and the tools to shift queued mail) - but definitely worth rechecking. -Jon
First
|
Prev
|
Pages: 1 2 Prev: O Poder do DOIS: Uma rota simples ao Sucesso... Responda para você mesmo essas perguntas: Você pode adquirir um Pacote Standard por mês por US$20 dólares? Você pode afiliar todos os meses DUAS pessoas que façam o mesmo que VOCÊ? BEM... Se você foc Next: Sendmail queue runner query - with deep queues starting from scratch... |