Interrupt Affinity in SMP
in [Kernel]
|
Prev: [RFC 2/3] mm: iommu: A physical allocator for the VCMM
Next: [ANNOUNCE] MDB Merkey's Kernel Debugger x86_64 2.6.34 06-29-2010 Release 2
From: Bryan Hundven on 19 Jul 2010 16:10 On Mon, Jul 19, 2010 at 12:22 PM, Robert Hancock <hancockrwd(a)gmail.com> wrote: > On Sat, Jul 17, 2010 at 2:02 PM, Bryan Hundven <bryanhundven(a)gmail.com> wrote: >> Please see the two attached examples. >> >> Notice on the 5410 example how we start with the affinity set to 0xff, >> and change it to 0xf0. >> This should spread the interrupts over the last 4 cores of this quad >> core - dual processor system. >> >> Also notice on the 5645 example, with the same commands we start with >> 0xffffff and change to 0xfff000 to spread the interrupts over the last >> 12 cores, but only the first of the last twelve cores receive >> interrupts. >> >> This is the inconsistency I was trying to explain before. > > As I mentioned before, I believe the behavior in this case is chipset > dependent, and potentially not something the kernel has control over. > In most cases distributing the same interrupt across multiple cores is > likely a bad idea in any case, unless the interrupt load is actually > so high that a single CPU can't handle it. Can anyone confirm that this is how the 5520 and newer xeon chipsets handle affinity? I might be totally wrong, but I thought that RSS (Receive Side Scaling, which is available on the 82576 network card in that 5645 xeon example) helps in that scenario? --Bryan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Robert Hancock on 19 Jul 2010 17:40
On Mon, Jul 19, 2010 at 2:03 PM, Bryan Hundven <bryanhundven(a)gmail.com> wrote: > On Mon, Jul 19, 2010 at 12:22 PM, Robert Hancock <hancockrwd(a)gmail.com> wrote: >> On Sat, Jul 17, 2010 at 2:02 PM, Bryan Hundven <bryanhundven(a)gmail.com> wrote: >>> Please see the two attached examples. >>> >>> Notice on the 5410 example how we start with the affinity set to 0xff, >>> and change it to 0xf0. >>> This should spread the interrupts over the last 4 cores of this quad >>> core - dual processor system. >>> >>> Also notice on the 5645 example, with the same commands we start with >>> 0xffffff and change to 0xfff000 to spread the interrupts over the last >>> 12 cores, but only the first of the last twelve cores receive >>> interrupts. >>> >>> This is the inconsistency I was trying to explain before. >> >> As I mentioned before, I believe the behavior in this case is chipset >> dependent, and potentially not something the kernel has control over. >> In most cases distributing the same interrupt across multiple cores is >> likely a bad idea in any case, unless the interrupt load is actually >> so high that a single CPU can't handle it. > > Can anyone confirm that this is how the 5520 and newer xeon chipsets > handle affinity? > > I might be totally wrong, but I thought that RSS (Receive Side > Scaling, which is available on the 82576 network card in that 5645 > xeon example) helps in that scenario? It looks like that card gives you multiple interrupt vectors that can be serviced independently by multiple CPUs. However, each interrupt vector is likely to only be handled by one CPU at a time. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |