kqemu+amd64: fpudna in kernel mode
in [Ports]
|
Prev: fusefs-kmod-0.3.0_3 requires the Kernel source to be installed
Next: Apache-2.2.4_2 : Undefined symbol "__h_errno"
From: Andriy Gapon on 11 Apr 2007 11:26 I get swarms of messages "fpudna in kernel mode" in system log when I run qemu with kqemu on FreeBSD 6.2 amd64 host with Windows XP 32-bit guest. I noticed that someone reported this problem a while ago: http://lists.freebsd.org/pipermail/freebsd-emulation/2006-February/001800.html There is no harm done, but the messages are annoying and they might be indicative of some bigger issue. Note that I am running qemu like follows: $ qemu-system-x86_64 ... -m 512 -net tap -net nic -localtime there is no -kernel-kqemu option and "info kqemu" in qemu monitor says that it is enabled for user code. As I understand "fpudna" means "FPU device not available" and the message comes from here: http://fxr.watson.org/fxr/source/amd64/amd64/trap.c on T_DNA ("device not avaialble") _kernel_ fault. BTW, not complaining about the following, but I think I should report it. When I use -kernel-kqemu option I get random crashes of the guest XP (BSOD) with diagnostics like the following (they vary): IRQL_NOT_GREATER_OR_EQUAL PFN_LIST_CORRUPT PAGE_FAULT_IN_NONPAGED_AREA Looks like all three are caused by some sort of memory corruption or driver misbehavior according to the following: http://aumha.org/a/stop.htm -- Andriy Gapon _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Jung-uk Kim on 11 Apr 2007 13:25 On Wednesday 11 April 2007 11:22 am, Andriy Gapon wrote: > I get swarms of messages "fpudna in kernel mode" in system log when > I run qemu with kqemu on FreeBSD 6.2 amd64 host with Windows XP > 32-bit guest. I noticed that someone reported this problem a while > ago: > http://lists.freebsd.org/pipermail/freebsd-emulation/2006-February/ >001800.html This happens because we do not allow FP instructions in kernel mode as the error message suggested. Executing FP instructions in kernel mode requires expensive save/restore cycles for FP states. > There is no harm done, but the messages are annoying and they might > be indicative of some bigger issue. > Note that I am running qemu like follows: > $ qemu-system-x86_64 ... -m 512 -net tap -net nic -localtime > > there is no -kernel-kqemu option and "info kqemu" in qemu monitor > says that it is enabled for user code. -kernel-kqemu option does not work for FreeBSD and it is very experimental feature for Linux x86-64. Since KQEMU is release under GPL recently, it can be fixed if someone's motivated enough, though. ;-) > As I understand "fpudna" means "FPU device not available" and the > message comes from here: > http://fxr.watson.org/fxr/source/amd64/amd64/trap.c > > on T_DNA ("device not avaialble") _kernel_ fault. You can ignore the complaints for now until we come up with better FP handling in kernel. > BTW, not complaining about the following, but I think I should > report it. When I use -kernel-kqemu option I get random crashes of > the guest XP (BSOD) with diagnostics like the following (they > vary): > > IRQL_NOT_GREATER_OR_EQUAL > PFN_LIST_CORRUPT > PAGE_FAULT_IN_NONPAGED_AREA > > Looks like all three are caused by some sort of memory corruption > or driver misbehavior according to the following: > http://aumha.org/a/stop.htm Windows guest is not supported by full virtualization mode, AFAIK. http://www.qemu.net/kqemu-doc.html BTW, it doesn't give you much performance gain as you may think: 'Best performances are achieved with Linux 2.4 kernels. Linux 2.6 works but the performance gains are small.' Even if we make it work with FreeBSD, I would expect similar results. Jung-uk Kim _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Bruce Evans on 12 Apr 2007 00:37 On Wed, 11 Apr 2007, Jung-uk Kim wrote: > On Wednesday 11 April 2007 11:22 am, Andriy Gapon wrote: >> I get swarms of messages "fpudna in kernel mode" in system log when >> I run qemu with kqemu on FreeBSD 6.2 amd64 host with Windows XP >> 32-bit guest. I noticed that someone reported this problem a while >> ago: >> http://lists.freebsd.org/pipermail/freebsd-emulation/2006-February/ >> 001800.html > > This happens because we do not allow FP instructions in kernel mode as > the error message suggested. Executing FP instructions in kernel > mode requires expensive save/restore cycles for FP states. Yes, kqemu is violating an invariant. This should cause a panic, but only causes a printf. In my kernel, it causes a debugger trap for the kernel T_DNA in trap() and sometimes a panic in npxdna() when a related invariant is violated, but I don't use kqemu and have never seen these happen. The panic in npxdna() rotted to a printf plus bogus error handling in rev.1.131 of npx.c. There may be no problem with kqemu, but it is or was hard to tell, since sources were unavailable and no one provided debugging info. I think kqemu doesn't use the FPU directly, but in does a software interrupt (int $0x10) to get to the T_DNA trap handler to emulate the corresponding trap (int $0x10 not in the instruction stream) that userland accessing the (not available) FPU would cause. This would give a perfect emulation in a fairly MI way if the software interrupt were actually supported, but even then it is slower than calling a (very MD) function that does the trap handling. In FreeBSD, calling npxdna() (fpudna() on amd64) directly would almost work, since npxdna() has no args -- in particular it doesn't depend on the trap frame. It does some (slightly broken due to races) invariants checking, so any hackish direct calls to it would have to avoid calling it if the invariant are not satified. The invariant is just that T_DNA doesn't happen if the thread already owns the FPU. In that case, kqemu shouldn't be doing a real or fake DNA trap to force the thread to own the FPU. Note that checking that the thread owns the FPU is nontrivial due to interrupts, and that even if it is done correctly, there is a race with interrupts (ithreads or preemptions trigger by interrupts) after becoming the owner, so there is no guarantee that a thread owns the FPU after doing a real or fake T_DNA to (try to) force the thread to own the FPU -- the ownership may change immediately after it is gained, before returning to userland or whatever actually uses the FPU. This is normally handled by simply repeating the T_DNA trap until the race is not lost. An emulator might need be careful with interrupts to get or implement the normal handling. The "slightly broken due to races" invariants checking in npxdna() is from not being careful with interrupts. Hmm, rev.1.131 made this more than slightly broken, since when the invariant PCPU_GET(fpcurthread) != NULL is violated due to PCPU_GET(fpcurthread) being curthread, the fixup does a delicate state change without keeping interrupts masked, so the new invariant PCPU_GET(fpcurthread) == curthread that goes with the T_DNA trap being turned off may be violated just before we turn the T_DNA trap off. Apart from the races, I think doing int $0x10's in the kernel is safe if and only if it is done near the syscall level. Then the result is the same as if userland did T_DNA's just before entering the kernel. (BTW, note that userland can't do software int $0x10's since these are privileged. AFAIR, there is no way to deny gate privilege via software interrupts to the kernel (ring 0). This usually isn't a problem since making software interrupts in the kernel is just foot shooting.) >> As I understand "fpudna" means "FPU device not available" and the >> message comes from here: >> http://fxr.watson.org/fxr/source/amd64/amd64/trap.c >> >> on T_DNA ("device not avaialble") _kernel_ fault. > > You can ignore the complaints for now until we come up with better FP > handling in kernel. He won't be able to ignore them when the handling is to panic :-). The other message "fpudna: fpucurthread == curthread %d times" is something to worry about now. Just printing that message enlarges the race window significantly. > Windows guest is not supported by full virtualization mode, AFAIK. > > http://www.qemu.net/kqemu-doc.html > > BTW, it doesn't give you much performance gain as you may think: > > 'Best performances are achieved with Linux 2.4 kernels. Linux 2.6 > works but the performance gains are small.' > > Even if we make it work with FreeBSD, I would expect similar results. Is that because 2.6 is already better? Is it a 2.6 host or target? Bruce _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Andriy Gapon on 12 Apr 2007 06:28 on 11/04/2007 20:07 Jung-uk Kim said the following: > On Wednesday 11 April 2007 11:22 am, Andriy Gapon wrote: >> I get swarms of messages "fpudna in kernel mode" in system log when >> I run qemu with kqemu on FreeBSD 6.2 amd64 host with Windows XP >> 32-bit guest. I noticed that someone reported this problem a while >> ago: >> http://lists.freebsd.org/pipermail/freebsd-emulation/2006-February/ >> 001800.html > > This happens because we do not allow FP instructions in kernel mode as > the error message suggested. Executing FP instructions in kernel > mode requires expensive save/restore cycles for FP states. > Jung-uk, thank you for the reply but there is still something not clear to me. I've read Bruce's response also, but I must admit I couldn't understand much. So, if floating point instructions cause trap then they apparently fail (or no ?), then why does (k)qemu executes them anyway ? And how (k)qemu is able to proceed normally after such failure ? -- Andriy Gapon _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Jung-uk Kim on 12 Apr 2007 14:22
> On Thursday 12 April 2007 12:32 am, Bruce Evans wrote: > > On Wed, 11 Apr 2007, Jung-uk Kim wrote: > He won't be able to ignore them when the handling is to panic :-). What I meant was more like "It is not trivial to fix and nobody is motivated enough to fix it yet. Sorry, ignore it for now." I thought "KQEMU/FreeBSD is borken" was too harsh. ;-) > > Even if we make it work with FreeBSD, I would expect similar > > results. > > Is that because 2.6 is already better? Dunno. I am not a Linux hacker. ;-) > Is it a 2.6 host or target? Target. BTW, there are many OSs out there that they allow FP operations in kernel including Windows, Mac OS X, DragonFlyBSD, etc. Even ndis(4) causes lots of fpudna warnings with some Windows drivers. Since there is no driver for it, I am stuck with ndis(4). Same thing is true for QEMU, i.e., I have no alternative. Bochs does not support FreeBSD/amd64 guest (no PAT emulation), VMware does not support FreeBSD host, Xen is far from dom0 support, etc. If someone's motivated enough, we can borrow some code from DragonFlyBSD: http://lists.freebsd.org/pipermail/freebsd-arch/2004-June/002271.html Jung-uk Kim _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org" |