Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Ingo Molnar on 18 Mar 2010 18:50 * Zachary Amsden <zamsden(a)redhat.com> wrote: > On 03/18/2010 11:15 AM, Ingo Molnar wrote: > >* Zachary Amsden<zamsden(a)redhat.com> wrote: > > > >>On 03/18/2010 12:50 AM, Ingo Molnar wrote: > >>>* Avi Kivity<avi(a)redhat.com> wrote: > >>> > >>>>>The moment any change (be it as trivial as fixing a GUI detail or as > >>>>>complex as a new feature) involves two or more packages, development speed > >>>>>slows down to a crawl - while the complexity of the change might be very > >>>>>low! > >>>>Why is that? > >>>It's very simple: because the contribution latencies and overhead compound, > >>>almost inevitably. > >>> > >>>If you ever tried to implement a combo GCC+glibc+kernel feature you'll know > >>>... > >>> > >>>Even with the best-run projects in existence it takes forever and is very > >>>painful - and here i talk about first hand experience over many years. > >>Ingo, what you miss is that this is not a bad thing. Fact of the > >>matter is, it's not just painful, it downright sucks. > >Our experience is the opposite, and we tried both variants and report about > >our experience with both models honestly. > > > >You only have experience about one variant - the one you advocate. > > > >See the assymetry? > > > >>This is actually a Good Thing (tm). It means you have to get your > >>feature and its interfaces well defined and able to version forwards > >>and backwards independently from each other. And that introduces > >>some complexity and time and testing, but in the end it's what you > >>want. You don't introduce a requirement to have the feature, but > >>take advantage of it if it is there. > >> > >>It may take everyone else a couple years to upgrade the compilers, > >>tools, libraries and kernel, and by that time any bugs introduced by > >>interacting with this feature will have been ironed out and their > >>patterns well known. > >Sorry, but this is pain not true. The 2.4->2.6 kernel cycle debacle has taught > >us that waiting long to 'iron out' the details has the following effects: > > > > - developer pain > > - user pain > > - distro pain > > - disconnect > > - loss of developers, testers and users > > - grave bugs discovered months (years ...) down the line > > - untested features > > - developer exhaustion > > > >It didnt work, trust me - and i've been around long enough to have suffered > >through the whole 2.5.x misery. Some of our worst ABIs come from that cycle as > >well. > > You're talking about a single project and comparing it to my argument about > multiple independent projects. In that case, I see no point in the > discussion. If you want to win the argument by strawman, you are welcome to > do so. The kernel is a very complex project with many ABI issues, so all those arguments apply to it as well. The description you gave: | This is actually a Good Thing (tm). It means you have to get your feature | and its interfaces well defined and able to version forwards and backwards | independently from each other. And that introduces some complexity and | time and testing, but in the end it's what you want. You don't introduce a | requirement to have the feature, but take advantage of it if it is there. matches the kernel too. We have many such situations. (Furthermore, the tools/perf/ situation, which relates to ABIs and user-space/kernel-space interactions is similar as well.) Do you still think i'm making a straw-man argument? > > Sorry, but i really think you are really trying to rationalize a > > disadvantage here ... > > This could very well be true, but until someone comes forward with > compelling numbers (as in, developers committed to working on the project, > number of patches and total amount of code contribution), there is no point > in having an argument, there really isn't anything to discuss other than > opinion. My opinion is you need a really strong justification to have a > successful fork and I don't see that justification. I can give you rough numbers for tools/perf - if that counts for you. For the first four months of its existence, when it was a separate project, i had a single external contributor IIRC. The moment it went into the kernel repo the number of contributors and contributions skyrocketed and basically all contributions were top-notch. We are at 60+ separate contributors now (after about 8 months upstream) - which is still small compared to the kernel or to Qemu, but huge for a relatively isolated project like instrumentation. So in my estimation tools/kvm/ would certainly be popular. Whether it would be more popular than current Qemu is hard to tell - it would be pure speculation. Any reliable numbers for the other aspect, whether a split project creates a more fragile and less developed ABI would be extremely hard to get. I believe it to be true, but that's my opinion based on my experience with other projects, extrapolated to KVM/Qemu. Anyway, the issue is moot as there's clear opposition to the unification idea. Too bad - there was heavy initial opposition to the arch/x86 unification as well [and heavy opposition to tools/perf/ as well], still both worked out extremely well :-) Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 19 Mar 2010 03:30 On 03/19/2010 12:44 AM, Ingo Molnar wrote: > > Too bad - there was heavy initial opposition to the arch/x86 unification as > well [and heavy opposition to tools/perf/ as well], still both worked out > extremely well :-) > Did you forget that arch/x86 was a merging of a code fork that happened several years previously? Maybe that fork shouldn't have been done to begin with. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 19 Mar 2010 04:00 On 03/18/2010 07:28 PM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >> On 03/18/2010 07:02 PM, Ingo Molnar wrote: >> >>> I find the 'KVM mostly cares about the server, not about the desktop' >>> attitude expressed in this thread troubling. >>> >> It's not kvm, just it's developers (and their employers, where applicable). >> If you post desktop oriented patches I'm sure they'll be welcome. >> > Just such a patch-set was posted in this very thread: 'perf kvm'. > > There were two negative reactions immediately, both showed a fundamental > server versus desktop bias: > > - you did not accept that the most important usecase is when there is a > single guest running. > Well, it isn't. > - the reaction to the 'how do we get symbols out of the guest' sub-question > was, paraphrased: 'we dont want that due to<unspecified> security threat > to XYZ selinux usecase with lots of guests'. > When I review a patch, I try to think of the difficult cases, not just the easy case. > Anyone being aware of how Linux and KVM is being used on the desktop will know > how detached that attitude is from the typical desktop usecase ... > > Usability _never_ sucks because of lack of patches or lack of suggestions. I > bet if you made the next server feature contingent on essential usability > fixes they'd happen overnight - for God's sake there's been 1000 commits in > the last 3 months in the Qemu repository so there's plenty of manpower... > First of all I am not a qemu maintainer. Second, from my point of view all contributors are volunteers (perhaps their employer volunteered them, but there's no difference from my perspective). Asking them to repaint my apartment as a condition to get a patch applied is abuse. If a patch is good, it gets applied. > Usability suckage - and i'm not going to be popular for saying this out loud - > almost always shows a basic maintainer disconnect with the real world. See > your very first reactions to my 'KVM usability' observations. Read back your > and Anthony's replies: total 'sure, patches welcome' kind of indifference. It > is _your project_, not some other project down the road ... > I could drop everything and write a gtk GUI for qemu. Is that what you want? If someone is truly interested in a qemu usability, it's up to them to write the patches. Personally I've never missed the eject button. As to disconnect from the real world, most products based on kvm and qemu (and Linux) are server based. Perhaps that's the reason people emphasise that? Maybe if Linux had 10-20% desktop market penetration, there would be more interest in a bells and whistles qemu GUI. > So that is my first-hand experience about how you are welcoming these desktop > issues, in this very thread. I suspect people try a few times with > suggestions, then get shot down like our suggestions were shot down and then > give up. > I don't recall anyone trying this much less being shot down. Perhaps people are concentrating on virt-manager and the like and leaving qemu alone. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 19 Mar 2010 05:00 * Avi Kivity <avi(a)redhat.com> wrote: > > There were two negative reactions immediately, both showed a fundamental > > server versus desktop bias: > > > > - you did not accept that the most important usecase is when there is a > > single guest running. > > Well, it isn't. Erm, my usability points are _doubly_ true when there are multiple guests ... The inconvenience of having to type: perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms \ --guestmodules=/home/ymzhang/guest/modules top is very obvious even with a single guest. Now multiply that by more guests ... The crux is: we are working on improving KVM instrumentation. There are working patches posted to this thread and we would like to have/implement an automatism to allow the discovery of all this information. The information should be available to the developer who wants it, and easily/transparently so - in true Linux fashion. > > - the reaction to the 'how do we get symbols out of the guest' sub-question > > was, paraphrased: 'we dont want that due to<unspecified> security threat > > to XYZ selinux usecase with lots of guests'. > > When I review a patch, I try to think of the difficult cases, not > just the easy case. You havent articulated an actionable reason and you have suggested no solution either, you just passive-agressive backed the claim that giving developers access to the symbol space is some sort of vague 'security threat'. If that is not so i'd be glad to be proven wrong. > > Anyone being aware of how Linux and KVM is being used on the desktop will > > know how detached that attitude is from the typical desktop usecase ... > > > > Usability _never_ sucks because of lack of patches or lack of suggestions. > > I bet if you made the next server feature contingent on essential > > usability fixes they'd happen overnight - for God's sake there's been 1000 > > commits in the last 3 months in the Qemu repository so there's plenty of > > manpower... > > First of all I am not a qemu maintainer. [...] That is the crux of the matter. My experience in these threads was that no-one really seems to feel in charge of the whole thing. Should we really wonder why KVM usability sucks? > [...] Second, from my point of view all contributors are volunteers (perhaps > their employer volunteered them, but there's no difference from my > perspective). Asking them to repaint my apartment as a condition to get a > patch applied is abuse. If a patch is good, it gets applied. This is one of the weirdest arguments i've seen in this thread. Almost all the time do we make contributions conditional on the general shape of the project. Developers dont get to do just the fun stuff. This is a basic quid pro quo: new features introduce risks and create additional workload not just to the originating developer but on the rest of the community as well. You should check how Linus has pulled new features in the past 15 years: he very much requires the existing code to first be top-notch before he accepts new features for a given area of functionality. Doing that and insisting on developers to see those imbalances as well is absolutely essential to code quality: otherwise everyone would be running around implementing just the features they are interested in, without regard for the general health of the project. Of course, if you keep the project in two halves (KVM and Qemu), and pretend that they are separate and have little relation, imbalances of quality can mount up and you can throw your hands up and say that it's "too bad, I'm not maintaining that". It is your basic duty as a Linux maintainer to keep balances of quality. I do it all day, other maintainers do it all day. > > Usability suckage - and i'm not going to be popular for saying this out > > loud - almost always shows a basic maintainer disconnect with the real > > world. See your very first reactions to my 'KVM usability' observations. > > Read back your and Anthony's replies: total 'sure, patches welcome' kind > > of indifference. It is _your project_, not some other project down the > > road ... > > I could drop everything and write a gtk GUI for qemu. Is that what you > want? No, my suggestion to you (it's up to you whether you give my opinion any weight) is to accept your mistakes and improve, and to not stand in the way of people who'd like to improve the situation. You are happy with the server features and you also made it clear that you dont feel responsible for the rest of the package - which is a big mistake IMO. Also, you have demonstrated it in this thread that you have near zero technical clue about basic desktop and development usability matters - for example your stance on symbol space access and your stance on how to enumerate guests symbolically are outright bizarre. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 19 Mar 2010 09:10
On 03/19/2010 03:53 AM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >>> There were two negative reactions immediately, both showed a fundamental >>> server versus desktop bias: >>> >>> - you did not accept that the most important usecase is when there is a >>> single guest running. >>> >> Well, it isn't. >> > Erm, my usability points are _doubly_ true when there are multiple guests ... > > The inconvenience of having to type: > > perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms \ > --guestmodules=/home/ymzhang/guest/modules top > > is very obvious even with a single guest. Now multiply that by more guests ... > If you want to improve this, you need to do the following: 1) Add a userspace daemon that uses vmchannel that runs in the guest and can fetch kallsyms and arbitrary modules. If that daemon lives in tools/perf, that's fine. 2) Add a QMP interface in qemu to interact with such daemon 3) Add a default QMP port in a well known location[1] 4) Modify the perf tool to look for a default QMP port. In the case of a single guest, there's one port. If there are multiple guests, then you will have to connect to each port, find the name or any other identifying information, and let the user choose. Patches are certainly welcome. [1] I've written up this patch and will send it out some time today. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |