Prev: [PATCH 2/4] panic: Add taint flag TAINT_FIRMWARE_WORKAROUND ('I')
Next: [PATCH] raw: fsync method is now required
From: Dominik Brodowski on 3 Apr 2010 18:40
Hey!

Before I'm off hiding some Easter eggs, here are some questions and
issues related to "dynticks", NOHZ, and powertop:

1) single-CPU systems, SMP-capable kernel and RCU
2) dual-core CPU[*] and select_nohz_load_balancer()
3) USB, autosuspend failure, excessive ticks
4) SynPS/2 touchpad and hundreds of IRQs per second
5) powertop: 1 + 1 = 1


1) single-CPU systems, SMP-capable kernel and RCU

CONFIG_TREE_RCU=y
CONFIG_RCU_FANOUT=64
CONFIG_RCU_FAST_NO_HZ=y

Booting a SMP-capable kernel with "nosmp", or manually offlining one CPU
(or -- though I haven't tested it -- booting a SMP-capable kernel on a
system with merely one CPU) means that in up to about half of the calls to
tick_nohz_stop_sched_tick() are aborted due to rcu_needs_cpu(). This is
quite strange to me: AFAIK, RCU is an excellent tool for SMP, but not really
needed for UP? And all updates seem to be local to the CPU anyway.
Therefore, I'd presume that rcu_needs_cpu() should return 0 on
one-CPU-systems. Or could RCU switch between TINY_RCU on UP and TREE_RCU on
SMP (using alternatives or whatever)?


2) dual-core CPU[*] and select_nohz_load_balancer()
[*] (Intel(R) Core(TM)2 Duo CPU T7250)

# CONFIG_SCHED_SMT is not set
CONFIG_SCHED_MC=y
CONFIG_SCHED_HRTICK=y

CONFIG_SCHED_MC is igored, as mc_capable() returns 0 on a one-socket,
dual-core system. Quite surprisingly, even under moderate load (~98.0% idle)
while writing this bugreport, up to half of the calls to
tick_nohz_stop_sched_tick() are aborted due to select_nohz_load_balancer(1):

if (atomic_read(&nohz.load_balancer) == -1) {
/* make me the ilb owner */
if (atomic_cmpxchg(&nohz.load_balancer, -1, cpu) == -1)
return 1;

I'm not really sure, but I guess this is caused by the following phenomenon
under minor load but still, every once in a while, parallel work for both
CPUs:

CPU #0 CPU #1

<active> <active>
<idle> <active>
tick_nohz_stop_sched_tick(1) <active>
select_nohz_load_balancer(1) <active>
=> becomes ilb owner <idle>
=> tick is not stopped tick_nohz_stop_sched_tick(1)
=> CPU goes to sleep for 1 tick => as it isn't the ILB owner, tick
<sleep for 1 tick> is stopped .
---> scheduler_tick() <sleeeeeeeep>
tick_nohz_stop_sched_tick(0)
<still idle>
tick_nohz_stop_sched_tick(1)
select_nohz_load_balancer(1)
=> is ilb owner, all CPUs idle,
may go to sleep.

If both CPUs have hardly anything to do, letting the _active_ CPU do ilb
allows us to enter deep sleep states earlier, and longer:

current ILB model (* = ILB)

tick ---------- tick -------- tick ----- IRQ
CPU0: active|IDLE(C2)--|*|IDLE (C3) |
CPU1: active....| IDLE (C3) |
core: .......???| C2 | C3 |

ILB-by-active-CPU-on-light-load:

tick ---------- tick -------- tick ----- IRQ
CPU0: active|IDLE(C3) |
CPU1: active....*| IDLE (C3) |
core: .......????| C3 |


3) USB: built-in UHCI and a built-in 0a5c:2101 Broadcom Corp. A-Link
BlueUsbA2 Bluetooth module; built-in EHCI and a built-in 0ac8:c302 Z-Star
Microelectronics Corp. Vega USB 2.0 Camera.

usbcore.autosuspend is enabled (= 2), of course.

Recent USB suspend statistics
Active Device name
100.0% USB device 7-1 : BCM92045NMD (Broadcom Corp)
100.0% USB device 1-2 : Vega USB 2.0 Camera. (Vimicro Corp.)
100.0% USB device usb7 : UHCI Host Controller (Linux 2.6.34-rc3 uhci_hcd)
100.0% USB device usb1 : EHCI Host Controller (Linux 2.6.34-rc3 ehci_hcd)

Booting into /bin/bash on a SMP kernel booted with "nosmp" leads to ~ 10
wakeups per second; disabling the cursor helps halfway (~ 5 wakeups); and
manually unbinding the USB host drivers from the USB host devices finally
lead to ~ 1.1 wakeups per second. What's keeping USB from suspending these
unused devices here?


4) SynPS/2 touchpad:
Why does moving the touchpad lead to sooo many IRQs? I can't look as fast
as the mouse pointer seems to get new data:
62,5% (473,1) <interrupt> : PS/2 keyboard/mouse/touchpad


5) powertop and hrtimer_start_range_ns (tick_sched_timer) on a SMP kernel
booted with "nosmp":

Wakeups-from-idle per second : 9.9 interval: 15.0s
....
48.5% ( 9.4) <kernel core> : hrtimer_start (tick_sched_timer)
26.1% ( 5.1) <kernel core> : cursor_timer_handler (cursor_timer_handler)
20.6% ( 4.0) <kernel core> : usb_hcd_poll_rh_status (rh_timer_func)
1.0% ( 0.2) <kernel core> : arm_supers_timer (sync_supers_timer_fn)
0.7% ( 0.1) <interrupt> : ata_piix
...

Accoding to http://www.linuxpowertop.org , the count in the brackets is how
many wakeups per seconds were caused by one source. Adding all _except_
48.5% ( 9.4) <kernel core> : hrtimer_start (tick_sched_timer)
up leads to the 9.9; adding also the 9.4 leads to 19.3 wakeups-from-idle per
second. However, http://www.linuxpowertop.org says:

> "Should "Wakeups-from-idle per second" equal the sum of the
> wakeups/second/core listed on the "Top causes for wakeups" list?
>
> It should be higher, since there are some causes for wakeups that are nearly
> impossible to detect by software."


Best, and Happy Easter,

Dominik
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Dmitry Torokhov on 3 Apr 2010 20:00
Hi Dominik,

On Sun, Apr 04, 2010 at 12:33:28AM +0200, Dominik Brodowski wrote:
>
> 4) SynPS/2 touchpad:
> Why does moving the touchpad lead to sooo many IRQs? I can't look as fast
> as the mouse pointer seems to get new data:
> 62,5% (473,1) <interrupt> : PS/2 keyboard/mouse/touchpad
>

80 pps @ 6 bytes/packet = 480 interrupts/sec.

You can try using psmouse.rate=40 to limit it to 40 pps which should
bring it to the rate of standard PS/2 mouse at the expense of
sensitivity...

--
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Dominik Brodowski on 4 Apr 2010 06:40
Hi Dmitry,

On Sat, Apr 03, 2010 at 04:53:26PM -0700, Dmitry Torokhov wrote:
> On Sun, Apr 04, 2010 at 12:33:28AM +0200, Dominik Brodowski wrote:
> >
> > 4) SynPS/2 touchpad:
> > Why does moving the touchpad lead to sooo many IRQs? I can't look as fast
> > as the mouse pointer seems to get new data:
> > 62,5% (473,1) <interrupt> : PS/2 keyboard/mouse/touchpad
> >
>
> 80 pps @ 6 bytes/packet = 480 interrupts/sec.
>
> You can try using psmouse.rate=40 to limit it to 40 pps which should
> bring it to the rate of standard PS/2 mouse at the expense of
> sensitivity...

Excellent. Maybe this could be added to the Tips&Tricks section at the
Powertop website?

I guess obtaining all 6 bytes at once is not really possible? It
seems a new byte only appears ~1.75 ms after the last one, at least on my
notebook, so waiting for this is not an option...

Best,
Dominik


PS: Dmitry, got something small for you in return:


[PATCH] i8042: spelling fix

Signed-off-by: Dominik Brodowski <linux(a)dominikbrodowski.net>

diff --git a/drivers/input/serio/i8042.c b/drivers/input/serio/i8042.c
index 9302ba0..f61233e 100644
--- a/drivers/input/serio/i8042.c
+++ b/drivers/input/serio/i8042.c
@@ -38,7 +38,7 @@ MODULE_PARM_DESC(noaux, "Do not probe or use AUX (mouse) port.");

static bool i8042_nomux;
module_param_named(nomux, i8042_nomux, bool, 0);
-MODULE_PARM_DESC(nomux, "Do not check whether an active multiplexing conrtoller is present.");
+MODULE_PARM_DESC(nomux, "Do not check whether an active multiplexing controller is present.");

static bool i8042_unlock;
module_param_named(unlock, i8042_unlock, bool, 0);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Dominik Brodowski on 4 Apr 2010 06:50
Hey,

On Sat, Apr 03, 2010 at 04:53:26PM -0700, Dmitry Torokhov wrote:
> On Sun, Apr 04, 2010 at 12:33:28AM +0200, Dominik Brodowski wrote:
> >
> > 4) SynPS/2 touchpad:
> > Why does moving the touchpad lead to sooo many IRQs? I can't look as fast
> > as the mouse pointer seems to get new data:
> > 62,5% (473,1) <interrupt> : PS/2 keyboard/mouse/touchpad
> >
>
> 80 pps @ 6 bytes/packet = 480 interrupts/sec.
>
> You can try using psmouse.rate=40 to limit it to 40 pps which should
> bring it to the rate of standard PS/2 mouse at the expense of
> sensitivity...

as a sidenote: if we know -- like here -- that the next IRQ will be issued
soon, in approximately 1.75 ms (well, at least on my system), might it make
sense to make tick_nohz_get_sleep_length() smarter to know about this?

Best,
Dominik
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Alan Stern on 4 Apr 2010 11:20
On Sun, 4 Apr 2010, Dominik Brodowski wrote:

> Booting a SMP-capable kernel with "nosmp", or manually offlining one CPU
> (or -- though I haven't tested it -- booting a SMP-capable kernel on a
> system with merely one CPU) means that in up to about half of the calls to
> tick_nohz_stop_sched_tick() are aborted due to rcu_needs_cpu(). This is
> quite strange to me: AFAIK, RCU is an excellent tool for SMP, but not really
> needed for UP?

I can't answer the real question here, not knowing enough about the RCU
implementation. However, your impression is wrong: RCU very definitely
_is_ useful and needed on UP systems. It coordinates among processes
(and interrupt handlers) as well as among processors.


> 3) USB: built-in UHCI and a built-in 0a5c:2101 Broadcom Corp. A-Link
> BlueUsbA2 Bluetooth module; built-in EHCI and a built-in 0ac8:c302 Z-Star
> Microelectronics Corp. Vega USB 2.0 Camera.
>
> usbcore.autosuspend is enabled (= 2), of course.
>
> Recent USB suspend statistics
> Active Device name
> 100.0% USB device 7-1 : BCM92045NMD (Broadcom Corp)
> 100.0% USB device 1-2 : Vega USB 2.0 Camera. (Vimicro Corp.)
> 100.0% USB device usb7 : UHCI Host Controller (Linux 2.6.34-rc3 uhci_hcd)
> 100.0% USB device usb1 : EHCI Host Controller (Linux 2.6.34-rc3 ehci_hcd)
>
> Booting into /bin/bash on a SMP kernel booted with "nosmp" leads to ~ 10
> wakeups per second; disabling the cursor helps halfway (~ 5 wakeups); and
> manually unbinding the USB host drivers from the USB host devices finally
> lead to ~ 1.1 wakeups per second. What's keeping USB from suspending these
> unused devices here?

Either the drivers don't support autosuspend or the devices aren't
enabled for autosuspend. By default, autosuspend is disabled for
(almost) all non-hub devices. You or your distribution must enable
it manually by doing:

echo auto >/sys/bus/usb/devices/.../power/level

If the driver doesn't support autosuspend then enabling it won't be
enough; you'll also have to unbind the driver from the device. The
easiest way to do this is to unconfigure the device:

echo 0 >/sys/bus/usb/devices/.../bConfigurationValue

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
 |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: [PATCH 2/4] panic: Add taint flag TAINT_FIRMWARE_WORKAROUND ('I')
Next: [PATCH] raw: fsync method is now required