kmemleak, cpu usage jump out of nowhere
in [Kernel]
|
From: Catalin Marinas on 14 Jul 2010 06:00 Zeno Davatz <zdavatz(a)gmail.com> wrote: > I got a new Intel core-8 i7 processor. > > I am on kernel uname -a > > Linux zenogentoo 2.6.35-rc5 #97 SMP Tue Jul 13 16:13:25 CEST 2010 i686 > Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux > > Sometimes in the middle of nowhere all of a sudden all of my 8-cores > are at 100% CPU usage and my machine really lags and hangs and is not > useable anymore. Some random process just grabs a bunch CPUs according > to htop. > > dmesg tell me that > > kmemleak: 38 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak) These may be related to the NO_BOOTMEM configuration (I sent a patch in a separate reply). But even when kmemleak scans the memory, it only uses a single thread and you should only see a single CPU going to 100%. I don't think kmemleak scanning can explain why all the 8 cores are going up to 100%. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zeno Davatz on 14 Jul 2010 06:10 On Wed, Jul 14, 2010 at 11:55 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: > Catalin Marinas wrote: >> >> On Wed, 2010-07-14 at 09:27 +0100, Zeno Davatz wrote: >>> >>> On Wed, Jul 14, 2010 at 10:05 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> >>> wrote: >>>> >>>> On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: >> >>>>> I am attaching you the file from /sys/kernel/debug/kmemleak >>>> >>>> Zeno, can you post your dmesg and .config, please? >>> >>> Sure, see attached files. >> >> It looks like NO_BOOTMEM is enabled. You can try the attached patch (I >> need to post it again on the list). >> >> >> kmemleak: Add support for NO_BOOTMEM configurations >> >> From: Catalin Marinas <catalin.marinas(a)arm.com> >> >> With commits 08677214 and 59be5a8e, alloc_bootmem()/free_bootmem() and >> friends use the early_res functions for memory management when >> NO_BOOTMEM is enabled. This patch adds the kmemleak calls in the >> corresponding code paths for bootmem allocations. >> >> Signed-off-by: Catalin Marinas <catalin.marinas(a)arm.com> >> Cc: Yinghai Lu <yinghai(a)kernel.org> >> Cc: H. Peter Anvin <hpa(a)zytor.com> > > Makes sense. > > Acked-by: Pekka Enberg <penberg(a)cs.helsinki.fi> > > Zeno, this should fix the kmemleak false positives but not the big pauses > you're seeing. Thank for this detailed info Pekka! I will not apply the patch at the moment. Will it be in with the next RC from Linus? Or do you recommend I apply it? What I want it is to tame the temper of Tux and restrict him from eating my CPU-donuts (cores) at random. I need them for other processes. ;) Best Zeno -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zeno Davatz on 14 Jul 2010 06:10 Dear Catalin On Wed, Jul 14, 2010 at 11:57 AM, Catalin Marinas <catalin.marinas(a)arm.com> wrote: > Zeno Davatz <zdavatz(a)gmail.com> wrote: >> I got a new Intel core-8 i7 processor. >> >> I am on kernel uname -a >> >> Linux zenogentoo 2.6.35-rc5 #97 SMP Tue Jul 13 16:13:25 CEST 2010 i686 >> Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux >> >> Sometimes in the middle of nowhere all of a sudden all of my 8-cores >> are at 100% CPU usage and my machine really lags and hangs and is not >> useable anymore. Some random process just grabs a bunch CPUs according >> to htop. >> >> dmesg tell me that >> >> kmemleak: 38 new suspected memory leaks (see /sys/kernel/debug/kmemleak) >> kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) >> kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak) >> kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) >> kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) >> kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > > These may be related to the NO_BOOTMEM configuration (I sent a patch in > a separate reply). > > But even when kmemleak scans the memory, it only uses a single thread > and you should only see a single CPU going to 100%. I don't think > kmemleak scanning can explain why all the 8 cores are going up to 100%. I am doing: /usr/src/my2.6> sudo patch -p1 < patch_catalin patching file mm/bootmem.c Hunk #1 FAILED at 435. Hunk #2 FAILED at 461. 2 out of 2 hunks FAILED -- saving rejects to file mm/bootmem.c.rej patching file mm/page_alloc.c Hunk #1 FAILED at 3659. 1 out of 1 hunk FAILED -- saving rejects to file mm/page_alloc.c.rej Any hints why it wont apply? Will this patch be in the next RC? Best Zeno -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Catalin Marinas on 14 Jul 2010 08:00 On Wed, 2010-07-14 at 11:04 +0100, Zeno Davatz wrote: > On Wed, Jul 14, 2010 at 11:57 AM, Catalin Marinas > <catalin.marinas(a)arm.com> wrote: > > Zeno Davatz <zdavatz(a)gmail.com> wrote: > >> Sometimes in the middle of nowhere all of a sudden all of my 8-cores > >> are at 100% CPU usage and my machine really lags and hangs and is not > >> useable anymore. Some random process just grabs a bunch CPUs according > >> to htop. > > > > These may be related to the NO_BOOTMEM configuration (I sent a patch in > > a separate reply). > > > > But even when kmemleak scans the memory, it only uses a single thread > > and you should only see a single CPU going to 100%. I don't think > > kmemleak scanning can explain why all the 8 cores are going up to 100%. > > I am doing: > > /usr/src/my2.6> sudo patch -p1 < patch_catalin > patching file mm/bootmem.c > Hunk #1 FAILED at 435. > Hunk #2 FAILED at 461. > 2 out of 2 hunks FAILED -- saving rejects to file mm/bootmem.c.rej > patching file mm/page_alloc.c > Hunk #1 FAILED at 3659. > 1 out of 1 hunk FAILED -- saving rejects to file mm/page_alloc.c.rej > > Any hints why it wont apply? Will this patch be in the next RC? The patch is against 2.6.35-rc4. I'll send it to Linus and hopefully it will get merged during rc. BTW, you can disable kmemleak scanning by doing: # echo scan=off > /sys/kernel/debug/kmemleak Do you still get that high CPU usage? -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zeno Davatz on 14 Jul 2010 08:00
On Wed, Jul 14, 2010 at 1:54 PM, Catalin Marinas <catalin.marinas(a)arm.com> wrote: > On Wed, 2010-07-14 at 11:04 +0100, Zeno Davatz wrote: >> On Wed, Jul 14, 2010 at 11:57 AM, Catalin Marinas >> <catalin.marinas(a)arm.com> wrote: >> > Zeno Davatz <zdavatz(a)gmail.com> wrote: >> >> Sometimes in the middle of nowhere all of a sudden all of my 8-cores >> >> are at 100% CPU usage and my machine really lags and hangs and is not >> >> useable anymore. Some random process just grabs a bunch CPUs according >> >> to htop. >> > >> > These may be related to the NO_BOOTMEM configuration (I sent a patch in >> > a separate reply). >> > >> > But even when kmemleak scans the memory, it only uses a single thread >> > and you should only see a single CPU going to 100%. I don't think >> > kmemleak scanning can explain why all the 8 cores are going up to 100%. >> >> I am doing: >> >> /usr/src/my2.6> sudo patch -p1 < patch_catalin >> patching file mm/bootmem.c >> Hunk #1 FAILED at 435. >> Hunk #2 FAILED at 461. >> 2 out of 2 hunks FAILED -- saving rejects to file mm/bootmem.c.rej >> patching file mm/page_alloc.c >> Hunk #1 FAILED at 3659. >> 1 out of 1 hunk FAILED -- saving rejects to file mm/page_alloc.c.rej >> >> Any hints why it wont apply? Will this patch be in the next RC? > > The patch is against 2.6.35-rc4. I'll send it to Linus and hopefully it > will get merged during rc. > > BTW, you can disable kmemleak scanning by doing: > > # echo scan=off > /sys/kernel/debug/kmemleak Thank you for the hint! > Do you still get that high CPU usage? Not at the moment. I disabled CONFIG_NO_BOOTMEM: and rebooted onto the new bzImage. No "CPU-bad-mood" at the moment. Best Zeno -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |