Analysing an ARM perf.data file on a x86-64 workstation
in [Kernel]
|
From: Arnaldo Carvalho de Melo on 19 Jan 2010 08:20 CCing linux-perf-users, this may be interesting when investigating similar problems for other arches. Thanks James, got the vmlinux file and and already found and fixed two bugs, patches sent CCed to you. Ok, now how does it look like? [acme(a)doppio linux-2.6-tip]$ perf report -i ~/iles/perf.data --vmlinux ~/iles/vmlinux Failed to open /lib/ld-2.8.so, continuing without symbols Failed to open /root/perf, continuing without symbols Failed to open /lib/libc-2.8.so, continuing without symbols # Samples: 40653353521 # # Overhead Command Shared Object Symbol # ........ ............... ................. ...... # 99.94% swapper c0020ac4 [k] 0x000000c0020ac4 0.04% perf c0068f48 [k] 0x000000c0068f48 0.01% perf libc-2.8.so [.] 0x00000000067198 0.01% sleep c00acb44 [k] 0x000000c00acb44 0.00% perf perf [.] 0x0000000000265c 0.00% perf ld-2.8.so [.] 0x00000000015fa8 0.00% sleep ld-2.8.so [.] 0x00000000008ff0 0.00% perf [kernel.kallsyms] [k] smp_call_function_single # # (For a higher level overview, try: perf report --sort comm,dso) # [acme(a)doppio linux-2.6-tip]$ So we manage to decode at least one kernel entry, yay! To make sure we really did it right, we can use -v to show the address too: [acme(a)doppio linux-2.6-tip]$ perf report -v -i ~/iles/perf.data --vmlinux ~/iles/vmlinux Failed to open /lib/ld-2.8.so, continuing without symbols Failed to open /root/perf, continuing without symbols Failed to open /lib/libc-2.8.so, continuing without symbols # Samples: 40653353521 # # Overhead Command Shared Object Symbol # ........ ............... ................. ...... # 99.94% swapper c0020ac4 0x00000000c0020ac4 ! [k] 0x000000c0020ac4 0.04% perf c0068f48 0x00000000c0068f48 ! [k] 0x000000c0068f48 0.01% perf /lib/libc-2.8.so 0x0000000000067198 K [.] 0x00000000067198 0.01% sleep c00acb44 0x00000000c00acb44 ! [k] 0x000000c00acb44 0.00% perf /root/perf 0x000000000000265c K [.] 0x0000000000265c 0.00% perf /lib/ld-2.8.so 0x0000000000015fa8 K [.] 0x00000000015fa8 0.00% sleep /lib/ld-2.8.so 0x0000000000008ff0 K [.] 0x00000000008ff0 0.00% perf [kernel.kallsyms] 0x00000000c0068f48 ! [k] smp_call_function_single # # (For a higher level overview, try: perf report --sort comm,dso) # [acme(a)doppio linux-2.6-tip]$ Ok, so smp_call_function_single is 0xc0068f48, lets see if that matches what is in the vmlinux symtab you provided me: [acme(a)doppio linux-2.6-tip]$ file ~/iles/vmlinux /home/acme/iles/vmlinux: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, not stripped [acme(a)doppio linux-2.6-tip]$ readelf -s ~/iles/vmlinux | grep smp_call_function_single 57421: c0068f18 60 FUNC GLOBAL DEFAULT 3 smp_call_function_single [acme(a)doppio linux-2.6-tip]$ Ok, it matches, as 0xc0068f18 <= 0xc0068f48 < (0xc0068f18 + 0x60). But what about the other addresses for [k]ernel space perf hits such as 0xc0020ac4, 0xc0068f48, etc? 0xc0020ac4 is around here: 385: 00000000 0 FILE LOCAL DEFAULT ABS process.c <SNIP> 411: c0020aa0 40 FUNC LOCAL DEFAULT 3 default_idle 412: c0020ac8 0 NOTYPE LOCAL DEFAULT 3 $a so its well possible that this is really a very idle machine, right? 0xc0068f48 is around here: 7997: 00000000 0 FILE LOCAL DEFAULT ABS up.c 7998: c0068f18 0 NOTYPE LOCAL DEFAULT 3 $a 7999: c0068f50 0 NOTYPE LOCAL DEFAULT 3 $d 0xc00acb44 around here: 13645: 00000000 0 FILE LOCAL DEFAULT ABS mmap.c <SNIP> 13679: c00aca98 216 FUNC LOCAL DEFAULT 3 __vma_link_file 13680: c00acb70 0 NOTYPE LOCAL DEFAULT 3 $a 13681: c00acb70 428 FUNC LOCAL DEFAULT 3 unmap_region We now need to try to investigate why these things are being misannotated in the symtab. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jamie Iles on 19 Jan 2010 09:00 Hi Arnaldo, On Tue, Jan 19, 2010 at 11:09:58AM -0200, Arnaldo Carvalho de Melo wrote: > CCing linux-perf-users, this may be interesting when investigating > similar problems for other arches. > > Thanks James, got the vmlinux file and and already found and fixed two > bugs, patches sent CCed to you. > > Ok, now how does it look like? Well, with both patches applied, I still don't see any kernel symbols in perf top (even the raw addresses) or resolved names in report if I specify a vmlinux or if one gets autodetected. For example, when keeping the CPU busy with 'dd if=/dev/zero of=/dev/null', If I record with 'perf record -af -- sleep 1' then: root(a)picopc7302:~# ./perf record -af -- sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.071 MB perf.data (~3092 samples) ] root(a)picopc7302:~# ./perf report --vmlinux=/tmp/vmlinux -v | head -10 no symbols found in /bin/busybox, maybe install a debug package? # Samples: 435596970 # # Overhead Command Shared Object Symbol # ........ ............... ...................... ...... # 81.14% dd c0020ac8 0x00000000c0020ac8 ! [k] 0x000000c0020ac8 9.26% perf c0071bfc 0x00000000c0071bfc ! [k] 0x000000c0071bfc 3.53% dd /bin/busybox 0x000000000001d6f4 K [.] 0x0000000001d6f4 1.55% sleep c006c3a0 0x00000000c006c3a0 ! [k] 0x000000c006c3a0 1.38% dd /lib/libc-2.8.so 0x00000000000b4a2c d [.] __GI___libc_read root(a)picopc7302:~# mv /boot/vmlinux /boot/vmlinux root(a)picopc7302:~# mv /boot/vmlinux /boot/v root(a)picopc7302:~# ./perf report -v | head -10 Looking at the vmlinux_path (5 entries long) no symbols found in /bin/busybox, maybe install a debug package? # Samples: 435596970 # # Overhead Command Shared Object Symbol # ........ ............... ...................... ...... # 32.57% dd [kernel.kallsyms] 0x00000000c02aaef0 k [k] _raw_spin_unlock_irqrestore 12.30% dd [kernel.kallsyms] 0x00000000c006b5c4 k [k] lock_acquire 6.23% dd [kernel.kallsyms] 0x00000000c006c220 k [k] lock_release 5.38% dd [kernel.kallsyms] 0x00000000c006b9f0 k [k] lock_acquired 5.04% dd [kernel.kallsyms] 0x00000000c0020ac8 k [k] vector_swi > [snip] > > But what about the other addresses for [k]ernel space perf hits such as > 0xc0020ac4, 0xc0068f48, etc? > > 0xc0020ac4 is around here: > > 385: 00000000 0 FILE LOCAL DEFAULT ABS process.c > <SNIP> > 411: c0020aa0 40 FUNC LOCAL DEFAULT 3 default_idle > 412: c0020ac8 0 NOTYPE LOCAL DEFAULT 3 $a > > so its well possible that this is really a very idle machine, right? Yes, the run I sent you was pretty much just idling away. Jamie -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Arnaldo Carvalho de Melo on 19 Jan 2010 09:40 Em Tue, Jan 19, 2010 at 01:51:45PM +0000, Jamie Iles escreveu: > Hi Arnaldo, > > On Tue, Jan 19, 2010 at 11:09:58AM -0200, Arnaldo Carvalho de Melo wrote: > > CCing linux-perf-users, this may be interesting when investigating > > similar problems for other arches. > > > > Thanks James, got the vmlinux file and and already found and fixed two > > bugs, patches sent CCed to you. > > > > Ok, now how does it look like? > Well, with both patches applied, I still don't see any kernel symbols in perf > top (even the raw addresses) or resolved names in report if I specify a > vmlinux or if one gets autodetected. What if you try to do what I did, i.e. to try to decode the perf.data file on your x86 workstation? Does specifying --vmlinux /path/to/arm-vmlinux gets the symbols decoded? - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|
Pages: 1 Prev: kernel cpufreq on intel core2quad Next: ram_buffer_extend_print |