Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Anthony Liguori on 18 Mar 2010 12:50 On 03/18/2010 11:28 AM, Ingo Molnar wrote: >>> There are all kernel space projects, going through Xorg would be a >>> horrible waste of performance for full-screen virtualization. It's fine >>> for the windowed or networked case (and good as a compatibility fallback), >>> but very much not fine for local desktop use. >>> > For the full-screen case (which is a very common mode of using a guest OS on > the desktop) there's not much of window management needed. You need to > save/restore as you switch in/out. > I don't think I've ever used full-screen mode with my VMs and I use virtualization on a daily basis. We hear very infrequently from users using full screen mode. >> 3D graphics virtualization is extremely difficult in the non-passthrough >> case. It really requires hardware support that isn't widely available today >> (outside a few NVIDIA chipsets). >> > Granted it's difficult in the general case. > > >>>> Xorg framebuffer driver doesn't implement any of the optimizations that the >>>> Linux framebuffer supports and the Xorg driver does not provide use the >>>> kernel's interfaces for providing update regions. >>>> >>>> Of course, we need to pull in X into the kernel to fix this, right? >>>> >>> FYI, this part of X has already been pulled into the kernel, it's called >>> DRM. If then it's being expanded. >>> >> It doesn't provide the things we need to a good user experience. You need >> things like an absolute input device, host driven display resize, RGBA >> hardware cursors. None of these go through DRI and it's those things that >> really provide the graphics user experience. >> > With KSM the display resize is in the kernel. KMS > Cursor management is not. Yet: i > think it would be a nice feature as the cursor could move even if Xorg is > blocked or busy with other things. > If it was all in the kernel, we'd try to support it. >>>> Any sufficiently complicated piece of software is going to interact with >>>> a lot of other projects. The solution is not to pull it all into one >>>> massive repository. It's to build relationships and to find ways to >>>> efficiently work with the various communities. >>>> >>> That's my whole point with this thread: the kernel side of KVM and qemu, >>> but all practical purposes should not be two 'separate communities'. They >>> should be one and the same thing. >>> >> I don't know why you keep saying this. The people who are in these >> "separate communities" keep claiming that they don't feel this way. >> > If you are not two separate communities but one community, then why do you go > through the (somewhat masochistic) self-punishing excercise of keeping the > project in two different pieces? > I don't see any actual KVM developer complaining about this so I'm not sure why you're describing it like this. > In a distant past Qemu was a separate project and KVM was just a newcomer who > used it for fancy stuff. Today as you say(?) the two communities are one and > the same. Why not bring it to its logical conclusion? > We lose a huge amount of users and contributors if we put QEMU in the Linux kernel. As I said earlier, a huge number of our contributions come from people not using KVM. >> I'm not just saying this to be argumentative. Many of the people in the >> community have thought this same thing, and tried it themselves, and we've >> all come to the same conclusion. >> >> It's certainly possible that we just missed the obvious thing to do but >> we'll never know that unless someone shows us. >> > I'm not aware of anyone in the past having attempted to move qemu to > tools/kvm/ in the uptream kernel repo, and having reported on the experiences > with such a contribution setup. (obviously it's not possible at all without > heavy cooperation and acceptance from you and Avi, so this will probably > remain a thought experiment forever) > We've tried to create a "clean" version of QEMU specifically for KVM. Moving it into tools/kvm would be the second step. We've all failed on the firs step. > If then you must refer to previous attempts to 'strip down' Qemu, right? Those > attempts didnt really solve the fundamental problem of project code base > separation. > If the problem is combining the two, I've sent you a patch that you can put into tip.git if you're so inclined. Regards, Anthony Liguori > Ingo > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 18 Mar 2010 13:00 On 03/18/2010 06:13 PM, Ingo Molnar wrote: > Currently there's a wall between kernel developers and user-space developers, > and there's somewhat of an element of fear and arrogance on both sides. For > efficient technology such walls needs torn down and people need a bit more > experience with each other's areas. > I think you're increasing the height of that wall by arguing that a userspace project cannot be successful because it's development process sucks and the only way to fix it is to put it into the kernel where people know so much better. Instead we kernel developers should listen to requirements from users, even if their code isn't in tools/. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 13:00 * Jes Sorensen <Jes.Sorensen(a)redhat.com> wrote: > On 03/18/10 15:22, Ingo Molnar wrote: > > > >* Jes Sorensen<Jes.Sorensen(a)redhat.com> wrote: > >>Both perf and oprofile are still relatively small projects in comparison to > >>QEMU. > > > >So is your argument that the unification does not make sense due to size? > >Would a smaller Qemu be more appropriate for this purpose? > > As I have stated repeatedly in this discussion, a unification would hurt the > QEMU development process because it would alienate a large number of QEMU > developers who are *not* Linux kernel users. I took a quick look at the qemu.git log and more than half of all recent contributions came from Linux distributors. So without KVM Qemu would be a much, much smaller project. It would be similar to how it was 5 years ago. > QEMU is a lot more complex than you let on. Please educate me then about the specifics. > >>Well I think we are just going to agree to disagree on this one. I am not > >>against merging projects where it makes sense, but in this particular case I > >>am strongly convinced the loss would be much greater than the gain. > > > >I wish you said that based on first hand negative experience with > >unifications, not based on just pure speculation. > > > >(and yes, i speculate too, but at least with some basis) > > You still haven't given us a *single* example of unification of something > that wasn't purely linked to the Linux kernel. perf/ oprofile is 100% linked > to the Linux kernel, QEMU is not. I wish you would actually look at what > users use QEMU for. As long as you continue to purely speculate on this, to > use your own words, your arguments are not holding up. The stats show that the huge increase in Qemu contributions over the past few years was mainly due to KVM. Do you claim it wasnt? What other projects make use of it and pay developers to work on it? > And you are not being consistent either. You have conveniently continue to > ignore my questions about why the file system tools are not to be merged > into the Linux kernel source tree? Sorry, i didnt comment on it because the answer is obvious: the file system tools and pretty much any Linux-exclusive tool (such as udev) should be moved there. The difference is that there's not much active development done in most of those tools so the benefits are probably marginal. Both Qemu and KVM is being developed very actively though, so development model inefficiencies show up. Anyway, i didnt think i'd step into such a hornet's nest by explaining what i see as KVM's biggest weakness today and how i suggest it to be fixed :-) If you dont agree with me, then dont do it - no need to get emotional about it. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 18 Mar 2010 13:10 On 03/18/2010 10:17 AM, Ingo Molnar wrote: > * Anthony Liguori<anthony(a)codemonkey.ws> wrote: > > >> On 03/18/2010 08:00 AM, Ingo Molnar wrote: >> >>>> [...] kvm in fact knows nothing about vga, to take your last example. >>>> [...] >>>> >>> Look at the VGA dirty bitmap optimization a'ka the KVM_GET_DIRTY_LOG >>> ioctl. >>> >>> See qemu/kvm-all.c's kvm_physical_sync_dirty_bitmap(). >>> >>> It started out as a VGA optimization (also used by live migration) and >>> even today it's mostly used by the VGA drivers - albeit a weak one. >>> >>> I wish there were stronger VGA optimizations implemented, copying the >>> dirty bitmap is not a particularly performant solution. (although it's >>> certainly better than full emulation) Graphics performance is one of the >>> more painful aspects of KVM usability today. >>> >> We have to maintain a dirty bitmap because we don't have a paravirtual >> graphics driver. IOW, someone needs to write an Xorg driver. >> >> Ideally, we could just implement a Linux framebuffer device, right? >> > No, you'd want to interact with DRM. > Using DRM doesn't help very much. You still need an X driver and most of the operations you care about (video rendering, window movement, etc) are not operations that need to go through DRM. 3D graphics virtualization is extremely difficult in the non-passthrough case. It really requires hardware support that isn't widely available today (outside a few NVIDIA chipsets). >> Xorg framebuffer driver doesn't implement any of the optimizations that the >> Linux framebuffer supports and the Xorg driver does not provide use the >> kernel's interfaces for providing update regions. >> >> Of course, we need to pull in X into the kernel to fix this, right? >> > FYI, this part of X has already been pulled into the kernel, it's called DRM. > If then it's being expanded. > It doesn't provide the things we need to a good user experience. You need things like an absolute input device, host driven display resize, RGBA hardware cursors. None of these go through DRI and it's those things that really provide the graphics user experience. >> Any sufficiently complicated piece of software is going to interact with a >> lot of other projects. The solution is not to pull it all into one massive >> repository. It's to build relationships and to find ways to efficiently >> work with the various communities. >> > That's my whole point with this thread: the kernel side of KVM and qemu, but > all practical purposes should not be two 'separate communities'. They should > be one and the same thing. > I don't know why you keep saying this. The people who are in these "separate communities" keep claiming that they don't feel this way. I'm not just saying this to be argumentative. Many of the people in the community have thought this same thing, and tried it themselves, and we've all come to the same conclusion. It's certainly possible that we just missed the obvious thing to do but we'll never know that unless someone shows us. > The thing is, writing up a DRM connector to a guest Linux OS could be done in > no time. It could be deployed to users in no time as well, with the proper > development model. > If this is true, please demonstrate it. Prove your point with patches and I'll happily turn around and do whatever I can to help out. > That after years and years of waiting proper GX support is _still_ not > implemented in KVM is really telling of the efficiency of development based on > such disjoint 'communities'. Maybe put up a committee as well to increase > efficiency? ;-) > Nah, instead we can just have a few hundred mail thread on the list. Otherwise we'd have to write patches and do other kinds of productive work. Regards, Anthony Liguori > Ingo > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 13:10
* Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: > On Thu, Mar 18, 2010 at 6:38 PM, Anthony Liguori <anthony(a)codemonkey.ws> wrote: > >>>> There are all kernel space projects, going through Xorg would be a > >>>> horrible waste of performance for full-screen virtualization. It's fine > >>>> for the windowed or networked case (and good as a compatibility > >>>> fallback), but very much not fine for local desktop use. > >> > >> For the full-screen case (which is a very common mode of using a guest OS > >> on the desktop) there's not much of window management needed. You need to > >> save/restore as you switch in/out. > > > > I don't think I've ever used full-screen mode with my VMs and I use > > virtualization on a daily basis. > > > > We hear very infrequently from users using full screen mode. > > Sorry for getting slightly off-topic but I find the above statement > interesting. > > I don't use virtualization on daily basis but a working, fully integrated > full-screen model with VirtualBox was the only reason I bothered to give VMs > a second chance. From my point of view, the user experience of earlier > versions (e.g. Parallels) was just too painful to live with. That's the same i do, and that's what i'm hearing from other desktop users as well. The moment you work seriously in a guest OS you often want to switch to it full-screen, to maximize screen real-estate and to reduce host GUI element distractions. If it's just casual use of a single app then windowed mode suffices (but in that case performance doesnt matter much to begin with). I find the 'KVM mostly cares about the server, not about the desktop' attitude expressed in this thread troubling. > /me crawls back to his hole now... /me should do that too - this discussion is not resulting in any positive result so it has become rather pointless. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |