2.6.35-rc3 deadlocks on semaphore operations
in [Kernel]
|
Prev: DRM / radeon / KMS: Fix hibernation regression related to radeon PM (was: Re: [Regression, post-2.6.34] Hibernation broken on machines with radeon/KMS and r300)
Next: [PATCH] x86, UV: make kdump avoid stack dumps
From: Christoph Lameter on 21 Jun 2010 10:40 On Sat, 19 Jun 2010, Manfred Spraul wrote: > Hello Christoph, > > On 06/18/2010 04:56 PM, Christoph Lameter wrote: > > One important fact that I forgot to mention: This is on a dual quad core > > dell box (8 hardware threads) with CONCURRENCY_LEVEL set to 9. > > > > - Which version of make do you use? That would be one suspect that might use > semop(). $ make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for x86_64-pc-linux-gnu > - If I run make -j5 bzImage, then no semaphore arrays are used. > Could you try to figure out which executable is doing the 'semop()' call? Looks like this was ld > Or: what's the output of ps xawwwww_ Need to do another run with a broken kernel for this. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Manfred Spraul on 21 Jun 2010 13:00 On 06/21/2010 04:31 PM, Christoph Lameter wrote: >> - If I run make -j5 bzImage, then no semaphore arrays are used. >> Could you try to figure out which executable is doing the 'semop()' call? >> > Looks like this was ld > > Ok, that doesn't help: $ grep semop -r binutils-2.20.51.0.2/ <nothing> My guess: On your system, something is preloaded with LD_PRELOAD, and that library hangs. I have no idea which library, and I don't know if it's a library bug or a kernel bug. -- Manfred -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Christoph Lameter on 21 Jun 2010 13:10 On Sat, 19 Jun 2010, Manfred Spraul wrote: > What are the semaphore values? > > Could you run > $ ipcs linux-2.6$ ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x00000000 0 root 777 135168 0 0x00000000 163841 root 777 135168 2 0x00000000 327682 christoph 600 393216 2 dest 0x00000000 360451 christoph 600 393216 2 dest 0x00000000 393220 christoph 600 393216 2 dest 0x00000000 425989 christoph 600 393216 2 dest 0x00000000 458758 christoph 600 393216 2 dest 0x00000000 491527 christoph 600 393216 2 dest 0x00000000 524296 christoph 600 393216 2 dest 0x00000000 557065 christoph 600 393216 2 dest 0x00000000 589834 christoph 600 393216 2 dest 0x00000000 622603 christoph 600 393216 2 dest 0x00000000 655372 christoph 600 393216 2 dest 0x00000000 688141 christoph 600 393216 2 dest 0x00000000 720910 christoph 600 393216 2 dest 0x00000000 753679 christoph 600 393216 2 dest 0x00000000 786448 christoph 600 4 2 dest 0xcbc384f8 819217 christoph 600 64528 1 0x00000000 851986 christoph 600 4 2 dest 0x00000000 884755 christoph 600 4 2 dest 0x00000000 917524 christoph 600 393216 2 dest 0x00000000 950293 christoph 600 393216 2 dest 0x00000000 5537814 christoph 666 31620 3 dest 0x00000000 5275672 christoph 666 376320 1 dest 0x00000000 5341209 christoph 666 376320 2 dest ------ Semaphore Arrays -------- key semid owner perms nsems 0x5e8387d1 65536 christoph 600 1 0xcbc384f8 32769 christoph 600 1 ------ Message Queues -------- key msqid owner perms used-bytes messages 0x5e8387cf 65536 christoph 600 0 0 0x5e8387d0 98305 christoph 600 0 0 > $ cat /proc/sysvipc/sem linux-2.6$ cat /proc/sysvipc/sem key semid perms nsems uid gid cuid cgid otime ctime 1585678289 65536 600 1 1000 1000 1000 1000 1277138991 1277138892 -876378888 32769 600 1 1000 1000 1000 1000 1277139467 1277138069 > $ ./getall 32768 -v > > getall is attached. linux-2.6$ ~/getall 65536 -v getall <id> [-v] found 1 semaphores. 0: 0 (cnt 10 zcnt 0) linux-2.6$ ~/getall 32769 -v getall <id> [-v] found 1 semaphores. 0: 1 (cnt 0 zcnt 0) The process is "dash". Could have something to do with "fakeroot" christoph(a)JTCHITW00139:/proc/27878$ cat maps 00400000-00418000 r-xp 00000000 08:21 253992 /bin/dash 00617000-00618000 r--p 00017000 08:21 253992 /bin/dash 00618000-00619000 rw-p 00018000 08:21 253992 /bin/dash 00619000-0061c000 rw-p 00000000 00:00 0 024b6000-024d7000 rw-p 00000000 00:00 0 [heap] 2b03c4959000-2b03c4979000 r-xp 00000000 08:21 6259766 /lib/ld-2.11.1.so 2b03c4979000-2b03c497b000 rw-p 00000000 00:00 0 2b03c4b79000-2b03c4b7a000 r--p 00020000 08:21 6259766 /lib/ld-2.11.1.so 2b03c4b7a000-2b03c4b7b000 rw-p 00021000 08:21 6259766 /lib/ld-2.11.1.so 2b03c4b7b000-2b03c4b7c000 rw-p 00000000 00:00 0 2b03c4b7c000-2b03c4b85000 r-xp 00000000 08:21 5447953 /usr/lib/libfakeroot/libfakeroot-sysv.so 2b03c4b85000-2b03c4d84000 ---p 00009000 08:21 5447953 /usr/lib/libfakeroot/libfakeroot-sysv.so 2b03c4d84000-2b03c4d85000 r--p 00008000 08:21 5447953 /usr/lib/libfakeroot/libfakeroot-sysv.so 2b03c4d85000-2b03c4d86000 rw-p 00009000 08:21 5447953 /usr/lib/libfakeroot/libfakeroot-sysv.so 2b03c4da4000-2b03c4f1c000 r-xp 00000000 08:21 6260729 /lib/libc-2.11.1.so 2b03c4f1c000-2b03c511c000 ---p 00178000 08:21 6260729 /lib/libc-2.11.1.so 2b03c511c000-2b03c5120000 r--p 00178000 08:21 6260729 /lib/libc-2.11.1.so 2b03c5120000-2b03c5121000 rw-p 0017c000 08:21 6260729 /lib/libc-2.11.1.so 2b03c5121000-2b03c5127000 rw-p 00000000 00:00 0 2b03c5127000-2b03c5129000 r-xp 00000000 08:21 6260732 /lib/libdl-2.11.1.so 2b03c5129000-2b03c5329000 ---p 00002000 08:21 6260732 /lib/libdl-2.11.1.so 2b03c5329000-2b03c532a000 r--p 00002000 08:21 6260732 /lib/libdl-2.11.1.so 2b03c532a000-2b03c532b000 rw-p 00003000 08:21 6260732 /lib/libdl-2.11.1.so 2b03c532b000-2b03c532d000 rw-p 00000000 00:00 0 7fff258b3000-7fff258d5000 rw-p 00000000 00:00 0 [stack] 7fff2598f000-7fff25990000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
From: Christoph Lameter on 21 Jun 2010 13:20 On Mon, 21 Jun 2010, Manfred Spraul wrote: > On 06/21/2010 04:31 PM, Christoph Lameter wrote: > > > - If I run make -j5 bzImage, then no semaphore arrays are used. > > > Could you try to figure out which executable is doing the 'semop()' call? > > > > > Looks like this was ld > > > > > Ok, that doesn't help: > $ grep semop -r binutils-2.20.51.0.2/ > <nothing> > > My guess: On your system, something is preloaded with LD_PRELOAD, and that > library hangs. > I have no idea which library, and I don't know if it's a library bug or a > kernel bug. No need to guess. The kernel build is run under fakeroot as evident from the command line I gave you. Fakeroot works fine with 2.6.34. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Christoph Lameter on 21 Jun 2010 13:20
Some more information that may be helpful: proc/27878$ strace -p 27878 Process 27878 attached - interrupt to quit semop(65536, {{0, -1, SEM_UNDO}}, 1 proc/27878$ ps xawwwww_ ERROR: Unsupported option (BSD syntax) ps alx shows a number of stuck processes: 0 1000 20121 17784 20 0 367040 14416 poll_s Sl ? 0:07 /usr/lib/chromium-browser/chromium-browser --type=plugin --plugin-path=/var/lib/flashplugin-installer/npwr 0 1000 23477 20121 20 0 120864 19940 poll_s Sl ? 0:07 /usr/lib/nspluginwrapper/i386/linux/npviewer.bin --plugin /usr/lib/flashplugin-installer/libflashplayer.so 0 1000 23531 16635 20 0 13928 2292 wait S+ pts/0 0:00 /usr/bin/make -j9 EXTRAVERSION=-rc3-broke ARCH=x86_64 modules 0 1000 24724 23531 20 0 17220 5444 wait S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=fs 0 1000 24817 23531 20 0 13512 1928 pipe_w S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers 0 1000 25374 23531 20 0 14256 2384 pipe_w S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=net 0 1000 27275 24817 20 0 13252 1560 wait S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/media 0 1000 27294 27275 20 0 13272 1576 pipe_w S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/media/dvb 0 1000 27623 27275 20 0 20720 8980 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/media/video 0 1000 27633 25374 20 0 16696 4984 pipe_w S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=net/sunrpc 0 1000 27691 24724 20 0 18400 6700 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=fs/xfs 0 1000 27693 27294 20 0 18252 6504 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/media/dvb/frontends 0 1000 27743 25374 20 0 15988 4244 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=net/tipc 0 1000 27744 24817 20 0 15016 3268 pipe_w S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/misc 0 1000 27754 25374 20 0 14832 3008 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=net/wimax 0 1000 27769 27633 20 0 14816 2836 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=net/sunrpc/auth_gss 0 1000 27839 27744 20 0 13932 2100 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/misc/c2port 0 1000 27847 27744 20 0 13996 2224 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/misc/cb710 0 1000 27866 27744 20 0 13772 1844 semtim S+ pts/0 0:00 /usr/bin/make -f scripts/Makefile.build obj=drivers/misc/eeprom 0 1000 27877 27693 20 0 0 0 exit Z+ pts/0 0:00 [sh] <defunct> 0 1000 27878 27623 20 0 8292 352 semtim S+ pts/0 0:00 /bin/sh -c { echo drivers/media/video/zr364xx.ko; echo drivers/media/video/zr364xx.o; } > .tmp_versions/zr1 1000 27879 2140 20 0 20900 4000 wait S+ pts/1 0:00 bash 0 1000 27905 2097 20 0 20900 4972 wait Ss pts/3 0:00 bash 0 1000 27997 27905 20 0 6588 756 - R+ pts/3 0:00 ps alx -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |