Big OOO, SpMT, and possible designs
|
From: Rick Jones on 21 May 2010 13:18 Morten Reistad <first(a)last.name> wrote: > When asterisk switches 12800 calls on 8 processors, asterisk itself > uses less than 100% of cpu on ONE processor, but Linux itself took > the other 7. Web servers behave likewise. That sounds like something the folks on the netdev list hosted on vger.kernel.org might like to hear more about. rick jones -- portable adj, code that compiles under more than one compiler these opinions are mine, all mine; HP might not want them anyway... :) feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...
From: nmm1 on 21 May 2010 13:39 In article <ht6f5f$ef$4(a)usenet01.boi.hp.com>, Rick Jones <rick.jones2(a)hp.com> wrote: >> >From the measurements we have done the important contention parts >> >happen in the OS kernel. That is the single most critical piece to >> >parallellise. > >> Yup. That's the next step. I am interested in the one after that, >> like any good academic :-) > >That's ironic - in my corner of the 'net at least a great deal of >effort was put into getting a kernel to scale and thus out of the >way and we are having to address the applications and their developers >:) Surprise, surprise :-) Regards, Nick Maclaren.
From: Morten Reistad on 21 May 2010 15:55 In article <ht6f5f$ef$4(a)usenet01.boi.hp.com>, Rick Jones <rick.jones2(a)hp.com> wrote: >> >From the measurements we have done the important contention parts >> >happen in the OS kernel. That is the single most critical piece to >> >parallellise. > >> Yup. That's the next step. I am interested in the one after that, >> like any good academic :-) > >That's ironic - in my corner of the 'net at least a great deal of >effort was put into getting a kernel to scale and thus out of the >way and we are having to address the applications and their developers >:) The performance in Linux and the common internet utilities is not bad. It is also amusing to see the difference between Linux and (Free|Open)BSD. Linux handles media shuffling very well, but there is a little latency (sub-millisecond, but still) that get in the way of the single-packet servers. On Linux we see the best results for asterisk, mysql, apache, but the BSDs give better performance for ser&friends, bind etc. It might just be that the Linux UDP stack needs some work. -- mrr
From: Morten Reistad on 21 May 2010 15:50 In article <ht6f8c$ef$5(a)usenet01.boi.hp.com>, Rick Jones <rick.jones2(a)hp.com> wrote: >Morten Reistad <first(a)last.name> wrote: >> When asterisk switches 12800 calls on 8 processors, asterisk itself >> uses less than 100% of cpu on ONE processor, but Linux itself took >> the other 7. Web servers behave likewise. > >That sounds like something the folks on the netdev list hosted on >vger.kernel.org might like to hear more about. Netdev? They seem swamped with devices and fixes, and have had several notices about "fixes only please". OK, someone has to do that work, so I don't mind. The low end network stuff is very good. We don't have complete instrumentation for the kernel running for these analysis runs, but from what we _can_ see we use a lot of time for the actual I/O (we _are_ seeing several gigabits worth of 200-byte packets). Without interrupt coalescing the "pc server architecture" seems to have ceiling of around 4000 calls; no matter how much processing, cache etc is available. These systems go into interrupt saturation, and become very unresponsive. With interrupt coalescing we saw interrupt loads get under control. We still see interrupts more or less fill one of the processors, though. Probably there is always something to do, always an interrupt to handle; but the system doesn't go into saturation anymore. This accounts for two out of 8 cpus. We see staggering amounts of context swithces, and the network code is very busy. We do not, however, see anything that should preclude further scaling until we are saturated with interrupts again. That should be somewhere around 100.000 calls. Asterisk easily handles 20+ cpus for itself, making threads as necessary. But then, there may be some locking issues in the network stack. I doubt that anyone else has pushed a standard LAMP server so hard regarding media streams. After all, we are at a level where a single 19" rack can handle all the telephone calls of 5 million people. -- mrr
From: Rick Jones on 21 May 2010 17:58
Morten Reistad <first(a)last.name> wrote: > In article <ht6f8c$ef$5(a)usenet01.boi.hp.com>, > Rick Jones <rick.jones2(a)hp.com> wrote: > >That sounds like something the folks on the netdev list hosted on > >vger.kernel.org might like to hear more about. > Netdev? They seem swamped with devices and fixes, and have had > several notices about "fixes only please". "Fixes only please" is a regular event at particular phases in the release process when they are getting a release candidate ready. Then there should be a message announcing the opening of the merge window. rick jones -- web2.0 n, the dot.com reunion tour... these opinions are mine, all mine; HP might not want them anyway... :) feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH... |