Prev: Linux Friendly NAS?
Next: Now, my Debian thinks I have SCSI for my old IDE/ATA drivesafter installing Kernel 2.6.32-4?
From: frerotjs on 28 Apr 2010 09:32
Hi,
I'm writing to you guys to see if you would have an idea with the
problem I'm currently experiencing. So far, on HP servers everything
seems to be working well as when I put "ffffffff" in smp_affinity proc
file for an irq every cpu is handling the interrupt request. But we
recently we acquired DELL PowerEdge r610 and it seems that even when
smp_affinity value is "ffffffff" only the first CPU is handling all
the interrupts as shown here:

[root(a)testfw3 ~]# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
0: 124666 0 0 0 0
0 0 0 IO-APIC-edge timer
1: 2 0 0 0 0
0 0 0 IO-APIC-edge i8042
3: 6 0 0 0 0
0 0 0 IO-APIC-edge serial
8: 1 0 0 0 0
0 0 0 IO-APIC-edge rtc0
9: 0 0 0 0 0
0 0 0 IO-APIC-fasteoi acpi
12: 4 0 0 0 0
0 0 0 IO-APIC-edge i8042
16: 5119 0 0 0 0
0 0 0 IO-APIC-fasteoi ioc0
17: 291 0 0 0 0
0 0 0 IO-APIC-fasteoi uhci_hcd:usb1
18: 0 0 0 0 0
0 0 0 IO-APIC-fasteoi uhci_hcd:usb2
19: 24 0 0 0 0
0 0 0 IO-APIC-fasteoi ehci_hcd:usb5
20: 0 0 0 0 0
0 0 0 IO-APIC-fasteoi uhci_hcd:usb4
21: 42 0 0 0 0
0 0 0 IO-APIC-fasteoi uhci_hcd:usb3,
ehci_hcd:usb6
23: 94 0 0 0 0
0 0 0 IO-APIC-fasteoi ata_piix
56: 0 0 0 0 0
0 0 0 PCI-MSI-edge aerdrv
57: 0 0 0 0 0
0 0 0 PCI-MSI-edge aerdrv
58: 0 0 0 0 0
0 0 0 PCI-MSI-edge aerdrv
59: 0 0 0 0 0
0 0 0 PCI-MSI-edge aerdrv
65: 3263 0 0 0 0
0 0 0 PCI-MSI-edge eth4
75: 404 0 0 0 0
0 0 0 PCI-MSI-edge eth5
85: 1276 0 0 0 0
0 0 0 PCI-MSI-edge eth6
95: 826 0 0 0 0
0 0 0 PCI-MSI-edge eth7
NMI: 0 0 0 0 0
0 0 0 Non-maskable interrupts
LOC: 6095 62050 47672 5995 5593
23057 9556 24939 Local timer interrupts
SPU: 0 0 0 0 0
0 0 0 Spurious interrupts
PMI: 0 0 0 0 0
0 0 0 Performance monitoring interrupts
PND: 0 0 0 0 0
0 0 0 Performance pending work
RES: 40 56 117 43 47
115 321 322 Rescheduling interrupts
CAL: 36 60 62 59 52
58 57 46 Function call interrupts
TLB: 20 15 7 41 135
47 126 61 TLB shootdowns
TRM: 0 0 0 0 0
0 0 0 Thermal event interrupts
THR: 0 0 0 0 0
0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0
0 0 0 Machine check exceptions
MCP: 3 3 3 3 3
3 3 3 Machine check polls


[root(a)testfw3 ~]# for i in `ls /proc/irq/*/smp_affinity`; do echo -n
"$i mask:"; cat $i;done
/proc/irq/0/smp_affinity mask:ffffffff
/proc/irq/10/smp_affinity mask:ffffffff
/proc/irq/11/smp_affinity mask:ffffffff
/proc/irq/12/smp_affinity mask:ffffffff
/proc/irq/13/smp_affinity mask:ffffffff
/proc/irq/14/smp_affinity mask:ffffffff
/proc/irq/15/smp_affinity mask:ffffffff
/proc/irq/16/smp_affinity mask:ffffffff
/proc/irq/17/smp_affinity mask:ffffffff
/proc/irq/18/smp_affinity mask:ffffffff
/proc/irq/19/smp_affinity mask:ffffffff
/proc/irq/1/smp_affinity mask:ffffffff
/proc/irq/20/smp_affinity mask:ffffffff
/proc/irq/21/smp_affinity mask:ffffffff
/proc/irq/23/smp_affinity mask:ffffffff
/proc/irq/2/smp_affinity mask:ffffffff
/proc/irq/3/smp_affinity mask:ffffffff
/proc/irq/4/smp_affinity mask:ffffffff
/proc/irq/56/smp_affinity mask:ffffffff
/proc/irq/57/smp_affinity mask:ffffffff
/proc/irq/58/smp_affinity mask:ffffffff
/proc/irq/59/smp_affinity mask:ffffffff
/proc/irq/5/smp_affinity mask:ffffffff
/proc/irq/65/smp_affinity mask:ffffffff
/proc/irq/6/smp_affinity mask:ffffffff
/proc/irq/75/smp_affinity mask:ffffffff
/proc/irq/7/smp_affinity mask:ffffffff
/proc/irq/85/smp_affinity mask:ffffffff
/proc/irq/8/smp_affinity mask:ffffffff
/proc/irq/95/smp_affinity mask:ffffffff
/proc/irq/9/smp_affinity mask:ffffffff

On this DELL r610 server I'm however able to change the affinity mask
so for a specified irq a specific cpu is used. But as soon as I set
the mask to use more than 1 CPU the changes doesn't apply.

[root(a)testfw3 ~]# cat /proc/irq/65/smp_affinity
ffffffff
[root(a)testfw3 ~]# echo 2 > /proc/irq/65/smp_affinity
[root(a)testfw3 ~]# cat /proc/irq/65/smp_affinity
00000002
[root(a)testfw3 ~]# cat /proc/interrupts | egrep "^ 65|CPU"
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
65: 6610 336 0 0 0
0 0 0 PCI-MSI-edge eth4




[root(a)testfw3 ~]# echo 4 > /proc/irq/65/smp_affinity
[root(a)testfw3 ~]# cat /proc/interrupts | egrep "^ 65|CPU"
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
65: 6610 489 13 0 0
0 0 0 PCI-MSI-edge eth4



[root(a)testfw3 ~]# echo 6 > /proc/irq/65/smp_affinity
[root(a)testfw3 ~]# cat /proc/interrupts | egrep "^ 65|CPU"
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
65: 6610 489 194 0 0
0 0 0 PCI-MSI-edge eth4
[root(a)testfw3 ~]# cat /proc/interrupts | egrep "^ 65|CPU"
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
65: 6610 489 203 0 0
0 0 0 PCI-MSI-edge eth4
[root(a)testfw3 ~]# cat /proc/interrupts | egrep "^ 65|CPU"
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
65: 6610 489 215 0 0
0 0 0 PCI-MSI-edge eth4

Thank you for your input. And don't hesitate to point me to the right
place if you feel I should have written to another group.

Regards,
From: Evan24 on 7 May 2010 17:02
On Wed, 28 Apr 2010 06:32:53 -0700, frerotjs wrote:

> Hi,
> I'm writing to you guys to see if you would have an idea with the
> problem I'm currently experiencing. So far, on HP servers everything
> seems to be working well as when I put "ffffffff" in smp_affinity proc
> file for an irq every cpu is handling the interrupt request. But we
> recently we acquired DELL PowerEdge r610 and it seems that even when
> smp_affinity value is "ffffffff" only the first CPU is handling all the
> interrupts as shown here:
May I suggest checking out http://irqbalance.org/ ?
From: Chris Cox on 7 May 2010 17:31
On Wed, 2010-04-28 at 06:32 -0700, frerotjs wrote:
> Hi,
> I'm writing to you guys to see if you would have an idea with the
> problem I'm currently experiencing. So far, on HP servers everything
> seems to be working well as when I put "ffffffff" in smp_affinity proc
> file for an irq every cpu is handling the interrupt request. But we
> recently we acquired DELL PowerEdge r610 and it seems that even when
> smp_affinity value is "ffffffff" only the first CPU is handling all
> the interrupts as shown here:

no irqbalance running?

any kernel parms you are passing??



From: frerotjs on 17 May 2010 20:30
On May 7, 5:31 pm, Chris Cox <chrisnc...(a)endlessnow.com> wrote:
> On Wed, 2010-04-28 at 06:32 -0700,frerotjswrote:
> > Hi,
> >   I'm writing to you guys to see if you would have an idea with the
> > problem I'm currently experiencing. So far, on HP servers everything
> > seems to be working well as when I put "ffffffff" in smp_affinity proc
> > file for an irq every cpu is handling the interrupt request. But we
> > recently we acquired DELL PowerEdge r610 and it seems that even when
> > smp_affinity value is "ffffffff" only the first CPU is handling all
> > the interrupts as shown here:
>
> no irqbalance running?
>
> any kernel parms you are passing??

I use the Debian distribution no differences between kernels. Only
hardware change to the lastest HP and DELL servers, and no balancing...
 | 
Pages: 1
Prev: Linux Friendly NAS?
Next: Now, my Debian thinks I have SCSI for my old IDE/ATA drivesafter installing Kernel 2.6.32-4?