Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Andrea Arcangeli on 19 Mar 2010 11:00 Hi there, not really trying to get into the CC list of this discussion ;) but for what is worth I'd like to share my opinion on the matter. On Thu, Mar 18, 2010 at 09:44:18AM +0100, Jes Sorensen wrote: > What made KVM so successful was that the core kernel of the hypervisor > was designed the right way, as a kernel module where it belonged. It was > obvious to anyone who had been exposed to the main competition at the > time, Xen, that this was the right approach. What has ended up killing > Xen in the end is the not-invented-here approach of copying everything > over, reformatting it, and rewriting half of it, which made it > impossible to maintain and support as a single codebase. At my previous Full agreement with that. CVS/git/patches and development model is next to irrelevant compared to the basic design of the code. qemu (and especially qemu-kvm) is surely much closer to perf, than a firefox or openoffice, because there is some tight interconnect with the kernel API. And the skills required to produce useful patches in qemu are similar to the skills requires to produce useful patches for the kernel, more often than not a new feature in kvm also requires some merging of a qemu-kvm side patch (it always happened to me so far ;). But clearly we've to draw a barrier somewhere and while I could see things like systemtap and util-linux included into the kernel and perf already is, I've an hard time to see userland code supporting kernels other than linux into the kernel. I think that's probably where I'd draw the line. Let's say somebody creates a pure paravirt userland for kvm without full driver emulation that only runs on a linux kernel and no other OS, maybe that thing wouldn't be so controversial to include into the kernel as qemu is. qemu is clearly beyond the "only-running-on-a-linux-kernel" barrier... I'd definitely start with systemtap, which I think is even more suitable than perf to be merged into the kernel. Things useful only for developers like perf/systemtap makes even more sense to fetch silently hidden in a single pull. Those projects are so ideal to fetch together because you run your own compiled userland binary and not an rpm, and you need very latest kernel and userland package and sometime new userland might not work so well with older kernel too and the other way around. they're tool for developers and no developer cares about API as they rebuild latest userland code anyway, they almost don't require backwards compatibility of kernel. > So far your argument would justify pulling all of gdb into the kernel > git tree as well, to support the kgdb efforts, or gcc so we can get rid > of the gcc version quirks in the kernel header files, e2fsprogs and > equivalent for _all_ file systems included in the kernel so we can make > sure our fs tools never get out of sync with whats supported in the > kernel...... It also boils down to the maintainer, where the code is, defines the maintainer who pushes/commits it to the central repository that everyone pulls from. And having the code into Linus's tree doesn't make sense unless Linus is interested to follow and review qemu. So it'd only create blind pulls. But I entirely see what Ingo is going after and I have no doubt that contribution increases if some code is merged into the kernel even if it's userland. The more people clones a project, the more people builds, uses, reads the code and has incentive to contribute... and there's nothing else like the linux git tree to give visibility to a project and get more contributions (well as long as the pulled code requires similar skills to the kernel code of course). Plus it's annoying to go on web, find url to clone, clone.. running make is faster. But this purely is a PR effect. It's like free ads, to get more people using and looking into the code because it's already on the harddisk and you've only to run make. After somebody gets familiar with the pulled userland code because it find it in the tree and it didn't need to search the web and cut-and-paste and stuff, clone a new repo, I think it wouldn't matter anymore to have it into the kernel or not. Like for perf, by now I doubt it'd get less contributions if it's moved out of the kernel tree. The only reason to leave it there is if Linus actively checks the code before pulling it. So I think what would be nice maybe to get the PR/ads positive effect and get more _users_ (and later developers) involved without actually merging, is a command like: git clone linux-2.6 cd linux-2.6 tools/clone-project qemu-kvm tools/clone-project qemu tools/clone-project systemtap tools/clone-project seabios tools/clone-project e2fsprogs tools/clone-project perf ... And then maybe a git-send-email or similar command that would do the right thing and send it to the right list or similar. Learning the process of other projects is time consuming and requires some effort. But as far as qemu-kvm goes, the visibility is already there, and most people that could possibly contribute to the kernel side already has the userland cloned and pulls regularly from it so I doubt it'd generate anything remotely as beneficial like it did for perf. systemtap is really _identical_ to perf. You include it, lots more developers toys with it by just running "make", they find a bug and they fixes it, and they keep on contributing new features later after they got familiar with it. As usual ;). In separate mail Ingo said: > Btw., i made similar arguments to Avi about 3 years ago when it was > going upstream, that qemu should be unified with KVM. This is more > true today than ever. Well not sure if with KVM you mean qemu-kvm or KVM kernel code, but I would see an huge value and a win-win situation to see qemu-kvm and qemu unified. It's out of my reach how there can be still be a difference considering that nobody runs qemu with perhaps the exception of the maintainers themself. There's not even a qemu/kvm directory in qemu. These are the real problems that should be solved... Every time I've to send a patch I've to check if it also applies to qemu. I usually start with qemu-kvm to do my development there, and then I cross fingers and I hope the patch applies clean against qemu too, and send it there in that case. qemu and qemu-kvm are the same thing, it's the same people, it's the same community, it's the same skills, it's an absolute waste that a little amount of code isn't merged so we can all work on the same tree. And it's not an huge patch at all if compared to the size of qemu-kvm... so there can't be any technical explanation for qemu to take a tangent here. So my suggestion is to start with what will give a _real_ tangible benefit to developers (i.e. to all work on the same branch), the PR effect of merging it into the kernel would be minor for qemu-kvm, it really doesn't matter which url we pull the code from as long as there is only 1 url and not 2. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 20 Mar 2010 03:40 On 03/19/2010 10:53 AM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >>> There were two negative reactions immediately, both showed a fundamental >>> server versus desktop bias: >>> >>> - you did not accept that the most important usecase is when there is a >>> single guest running. >>> >> Well, it isn't. >> > Erm, my usability points are _doubly_ true when there are multiple guests ... > > The inconvenience of having to type: > > perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms \ > --guestmodules=/home/ymzhang/guest/modules top > > is very obvious even with a single guest. Now multiply that by more guests ... > Yes. That's why I asked how this is handled. > The crux is: we are working on improving KVM instrumentation. There are > working patches posted to this thread and we would like to have/implement an > automatism to allow the discovery of all this information. The information > should be available to the developer who wants it, and easily/transparently so > - in true Linux fashion. > > >>> - the reaction to the 'how do we get symbols out of the guest' sub-question >>> was, paraphrased: 'we dont want that due to<unspecified> security threat >>> to XYZ selinux usecase with lots of guests'. >>> >> When I review a patch, I try to think of the difficult cases, not >> just the easy case. >> > You havent articulated an actionable reason and you have suggested no solution > either, I did suggest a symbol server, and using a well-known location, though I'm unhappy with it. Multiple guest management should be done by the appropriate tools, not qemu or implicitly. > you just passive-agressive backed the claim that giving developers > access to the symbol space is some sort of vague 'security threat'. > Passive-aggressive? Should I see a doctor? > If that is not so i'd be glad to be proven wrong. > > >>> Anyone being aware of how Linux and KVM is being used on the desktop will >>> know how detached that attitude is from the typical desktop usecase ... >>> >>> Usability _never_ sucks because of lack of patches or lack of suggestions. >>> I bet if you made the next server feature contingent on essential >>> usability fixes they'd happen overnight - for God's sake there's been 1000 >>> commits in the last 3 months in the Qemu repository so there's plenty of >>> manpower... >>> >> First of all I am not a qemu maintainer. [...] >> > That is the crux of the matter. My experience in these threads was that no-one > really seems to feel in charge of the whole thing. I am comfortable with having someone I trust maintain qemu. While sometimes Anthony overrides me on issues where I know I'm right and he's wrong, still I prefer that to having to do everything myself, I would surely do a worse job due to overload. I you actually look at qemu patches, the vast majority have little to do directly with kvm; and I (along with Marcelo) maintain the kvm integration in qemu. > Should we really wonder why > KVM usability sucks? > That wouldn't change at all if I were to maintain it, since I wouldn't start writing a GUI for it and wouldn't force other contributors to do so as a condition for accepting unrelated patches. >> [...] Second, from my point of view all contributors are volunteers (perhaps >> their employer volunteered them, but there's no difference from my >> perspective). Asking them to repaint my apartment as a condition to get a >> patch applied is abuse. If a patch is good, it gets applied. >> > This is one of the weirdest arguments i've seen in this thread. Almost all the > time do we make contributions conditional on the general shape of the project. > Developers dont get to do just the fun stuff. > So, do you think a reply to a patch along the lines of NAK. Improving scalability is pointless while we don't have a decent GUI. I'll review you RCU patches _after_ you've contributed a usable GUI. ? > This is a basic quid pro quo: new features introduce risks and create > additional workload not just to the originating developer but on the rest of > the community as well. You should check how Linus has pulled new features in > the past 15 years: he very much requires the existing code to first be > top-notch before he accepts new features for a given area of functionality. > For a given area, yes. It makes sense to clean up code before changing it, otherwise cruft accumulates rapidly. What you're describing is completely different and amounts to total disregard of contributors' time and effort. > Doing that and insisting on developers to see those imbalances as well is > absolutely essential to code quality: otherwise everyone would be running > around implementing just the features they are interested in, without regard > for the general health of the project. > The general health of qemu in terms of code quality was indeed pretty bad and there was (and is) a massive effort to modernise it. If you're interested look at qdev and qmp. Both are efforts to improve the infrastructure rather than add features on rotten code, and very successful IMO. There was no effort to write a GUI since no one appears to be motivated to do it except you. > Of course, if you keep the project in two halves (KVM and Qemu), and pretend > that they are separate and have little relation, imbalances of quality can > mount up and you can throw your hands up and say that it's "too bad, I'm not > maintaining that". It is your basic duty as a Linux maintainer to keep > balances of quality. I do it all day, other maintainers do it all day. > IMO qemu quality has improved dramatically in the last year or two. >>> Usability suckage - and i'm not going to be popular for saying this out >>> loud - almost always shows a basic maintainer disconnect with the real >>> world. See your very first reactions to my 'KVM usability' observations. >>> Read back your and Anthony's replies: total 'sure, patches welcome' kind >>> of indifference. It is _your project_, not some other project down the >>> road ... >>> >> I could drop everything and write a gtk GUI for qemu. Is that what you >> want? >> > No, my suggestion to you (it's up to you whether you give my opinion any > weight) is to accept your mistakes and improve, and to not stand in the way of > people who'd like to improve the situation. You are happy with the server > features and you also made it clear that you dont feel responsible for the > rest of the package - which is a big mistake IMO. > If there were no capable maintainer I would reluctantly step in. That is not the case. If I were to displace Anthony then qemu quality would suffer, or I would have to drop kvm maintainership, or, if some false modesty is allowed, perhaps both. > Also, you have demonstrated it in this thread that you have near zero > technical clue about basic desktop and development usability matters > Neither do you. At least I have spent enough time among real usability people to know this. I don't have any pretences in this area and am happy to leave it to the experts. As infrastructure projects kvm and qemu should concentrate on providing flexible capabilities to consumers, which then expose it to users. These consumers can be server-oriented management applications, or end-user GUIs. My preferred plan for GUIs, btw, is a plugin based approach where qemu exposes its internal objects (the same ones that are exposed to management applications) to the GUI which can then manipulate them, without being co-maintained in the same code base. This allows multiple GUIs (KDE and GNOME) and allows people with a clue to work on them. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Andrea Arcangeli on 20 Mar 2010 11:10 On Fri, Mar 19, 2010 at 09:21:49AM +0200, Avi Kivity wrote: > On 03/19/2010 12:44 AM, Ingo Molnar wrote: > > > > Too bad - there was heavy initial opposition to the arch/x86 unification as > > well [and heavy opposition to tools/perf/ as well], still both worked out > > extremely well :-) > > > > Did you forget that arch/x86 was a merging of a code fork that happened > several years previously? Maybe that fork shouldn't have been done to > begin with. We discussed and probably timidly tried to share the sharable initially but we realized it was too time wasteful. In addition to having to adapt the code to 64bit we would also had to constantly solve another problem on top of it (see the various split on _32/_64, those takes time to achieve, maybe not huge time but still definitely some time and effort). Even in retrospect I am quite sure the way x86-64 happened was optimal and if we would go back we would do it again the exact same way even if the final object was to have a common arch/x86 (and thankfully Linus is flexible and smart enough to realize that code that isn't risking to destabilize anything shouldn't be forced out just because it's not to a totally theoretical-perfect-nitpicking-clean-state yet). It's still a lot of work do the unification later as a separate task, but it's not like if we did it immediately it would have been a lot less work. It's about the same amount of effort and we were able to defer it for later and decrease the time to market which surely has contributed to the success of x86-64. Problem of qemu is not some lack of GUI or that it's not included in the linux kernel git tree, the definitive problem is how to merge qemu-kvm/kvm and qlx into it. If you (Avi) were the qemu maintainer I am sure there wouldn't two trees so as a developer I would totally love it, and I am sure that with you as maintainer it would have a chance to move forward with qlx on desktop virtualization without proposing to extend vnc instead to achieve a "similar" result (imagine if btrfs is published on a website and people starts to discuss if it should ever be merged ever because reinventing some part of btrfs inside ext5 might achieve ""similar"" results). About a GUI for KVM to use on desktop distributions, that is an irrelevant concern compared to the lack of protocol more efficient than rdesktop/rdp/vnc for desktop virtualization. I've people asking me to migrate hundreds of desktops to desktop virtualization on KVM in their organizations and I tell them to use spice because I believe it's the most efficient option available (at least as far as we stick to open source open protocols), there are universities using spice on thousand of student desktops, and I think we need paravirt graphics to happen ASAP in the main qemu tree too. In short: running KVM on the desktop is irrelevant compared to running the desktop on KVM so I suggest to focus on what is more important first ;). Thanks, Andrea -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 21 Mar 2010 06:10 On 03/20/2010 04:59 PM, Andrea Arcangeli wrote: > On Fri, Mar 19, 2010 at 09:21:49AM +0200, Avi Kivity wrote: > >> On 03/19/2010 12:44 AM, Ingo Molnar wrote: >> >>> Too bad - there was heavy initial opposition to the arch/x86 unification as >>> well [and heavy opposition to tools/perf/ as well], still both worked out >>> extremely well :-) >>> >>> >> Did you forget that arch/x86 was a merging of a code fork that happened >> several years previously? Maybe that fork shouldn't have been done to >> begin with. >> > We discussed and probably timidly tried to share the sharable > initially but we realized it was too time wasteful. In addition to > having to adapt the code to 64bit we would also had to constantly > solve another problem on top of it (see the various split on _32/_64, > those takes time to achieve, maybe not huge time but still definitely > some time and effort). Even in retrospect I am quite sure the way > x86-64 happened was optimal and if we would go back we would do it > again the exact same way even if the final object was to have a common > arch/x86 (and thankfully Linus is flexible and smart enough to realize > that code that isn't risking to destabilize anything shouldn't be > forced out just because it's not to a totally > theoretical-perfect-nitpicking-clean-state yet). It's still a lot of > work do the unification later as a separate task, but it's not like if > we did it immediately it would have been a lot less work. It's about > the same amount of effort and we were able to defer it for later and > decrease the time to market which surely has contributed to the > success of x86-64. > In hindsight decisions are much easier. I agree it was less risky to fork than to share. But if another instruction set forks out a 64-bit not-exactly-compatible variant, I'm sure we'll start out shared and not fork it, especially if the platform remains the same. > Problem of qemu is not some lack of GUI or that it's not included in > the linux kernel git tree, the definitive problem is how to merge > qemu-kvm/kvm and qlx into it. If you (Avi) were the qemu maintainer I > am sure there wouldn't two trees so as a developer I would totally > love it, and I am sure that with you as maintainer it would have a > chance to move forward with qlx on desktop virtualization without > proposing to extend vnc instead to achieve a "similar" result (imagine > if btrfs is published on a website and people starts to discuss if it > should ever be merged ever because reinventing some part of btrfs > inside ext5 might achieve ""similar"" results). > The qemu/qemu-kvm fork is definitely hurting. Some history: when kvm started out I pulled qemu for fast hacking and, much like arch/x86_64, I couldn't destabilize qemu for something that was completely experimental (and closed source at the time). Moreover, it wasn't clear if the qemu community would be interested. The qemu-kvm fork was designed for minimal intrusion so I could merge upstream qemu regularly. This resulted in kvm integration that was fairly ugly. Later Anthony merged a well-integrated alternative implementation (in retrospect this was a mistake IMO - we were left with a well tested high performing ugly implementation and a clean, slow, untested, and unfeatured implementation, and no one who wants to merge the two). So now it is pretty confusing to read the code which has the two alternate implementation sometimes sharing code and sometimes diverging. > About a GUI for KVM to use on desktop distributions, that is an > irrelevant concern compared to the lack of protocol more efficient > than rdesktop/rdp/vnc for desktop virtualization. I've people asking > me to migrate hundreds of desktops to desktop virtualization on KVM in > their organizations and I tell them to use spice because I believe > it's the most efficient option available (at least as far as we stick > to open source open protocols), there are universities using spice on > thousand of student desktops, and I think we need paravirt graphics to > happen ASAP in the main qemu tree too. > That effort will have to wait for the spice project to mature. > In short: running KVM on the desktop is irrelevant compared to running > the desktop on KVM so I suggest to focus on what is more important > first ;). > Anyone can focus on what interests them, if someone has an interest in a good desktop-on-desktop experience they should start hacking and sending patches. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Gabor Gombas on 21 Mar 2010 09:30
On Thu, Mar 18, 2010 at 05:13:10PM +0100, Ingo Molnar wrote: > > Why does Linux AIO still suck? Why do we not have a proper interface in > > userspace for doing asynchronous file system operations? > > Good that you mention it, i think it's an excellent example. > > The suckage of kernel async IO is for similar reasons: there's an ugly package > separation problem between the kernel and between glibc - and between the apps > that would make use of it. No, kernel async IO sucks because it still does not play well with buffered I/O. Last time I checked (about a year ago or so), AIO syscall latencies were much worse when buffered I/O was used compared to direct I/O. Unfortunately, to achieve good performance with direct I/O, you need a HW RAID card with lots of on-board cache. Gabor -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |