Prev: [RFC] Tight check of pfn_valid on sparsemem
Next: sparc: move is_root_node from private header to of.h
From: Grant Likely on 12 Jul 2010 16:00
On Mon, Jul 12, 2010 at 1:34 PM, Linus Torvalds
<torvalds(a)linux-foundation.org> wrote:
> On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico(a)fluxnic.net> wrote:
>> I think Uwe could provide his script and add it to the kernel tree.
>> Then all architectures could benefit from it. �Having the defconfig
>> files contain only those options which are different from the defaults
>> is certainly more readable, even on x86.
>
> Quite possible. But maintainers would need to be on the lookout of
> people actually using the script, and refusing to apply patches that
> re-introduce the whole big thing.

I can (partially) speak for powerpc. If ARM uses this approach, then
I think we can do the same. After the defconfigs are trimmed, I
certainly won't pick up any more full defconfigs.

Of course, I'm also operating under the assumption that this is a
temporary measure until one of the better solutions is implemented. I
do suspect that the trimmed defconfigs will tend to be unstable and
will still require manual maintenance. I think the Kconfig fragments
approach is the most promising if the dependencies issue can be
solved.

g.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Uwe Kleine-König on 12 Jul 2010 16:10
Hi Linus,

On Mon, Jul 12, 2010 at 12:34:59PM -0700, Linus Torvalds wrote:
> On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico(a)fluxnic.net> wrote:
> >>
> >> � �Put another way: I realize that fairly late in the -rc series is
> >> actually a really good time to do this, rather than during the merge
> >> window itself when things are in flux. However, while it would be a
> >> good time to pull this for that reason, it's also a _horrible_ time to
> >> pull if it then regresses the defconfig uses, or if it causes horrible
> >> problems for linux-next merging etc.
> >
> > This cannot be any worse than wholesale removal of those files as you
> > were contemplating at some point. �Furthermore, on ARM we have someone
> > providing automatic rebuild of all defconfigs already, so any serious
> > issue should be noticed right away.
>
> You aren't really answering the question. The thing is, the wholesale
> removal wouldn't happen during the late -rc anyway, and it wouldn't
> cause any merge issues (merge conflicts where the file is just to be
> removed are trivial to take care of).
>
> So I'm really asking about the issue of "how does this work during the
> late -rc phase". Where the _timing_ is the important thing.
>
> Because I could as easily pull after 2.6.35 is out. That has it's own
> downsides (with all the merging going on), but at the same time, it's
> clearly the right time for a large change.
I hoped you to pull this in during the merge window, but doing it now is
OK for me, too.

> So when I ask about timing and "how does this work in late -rc", it's
> really about how safe it is to do now. Because if it isn't very
> obviously safe, I can't pull it.
>
> One part of the "obviously safe" thing is verification for each
> defconfig file that the end result is identical with the reduced
> version. Uwe implied that it was, and that was clearly the intention,
> but was there some explicit verification that yes, the
> before-and-after configurations are 100% the same?
I'm pretty sure it's 100% save as I only kick a line in the defconfig if
the result doesn't change. Well, modulo bugs in my script. But now
that it's public you can convince yourself it's (hopefully) correct.

> Also, I assume that while it's _mostly_ a transparent change to users,
> it does require that people do "make oldconfig" with the reduced
> defconfig file first, in order to avoid having to answer with the
> defaults. No?
No, $(make ..._defconfig) is enough to get the full .config.

> And the reduced defconfig files obviously do depend on
> the default in the Kconfig files themselves, so it does have that kind
> of fairly subtle semantic change that may not be entirely obvious or
> easy to notice.
Right.

> So from a timing standpoint, I just want to be convinced that "late
> -rc" really is the right thing. I'm not entirely sure it is, even
> though I do see advantages.
>
> > I think Uwe could provide his script and add it to the kernel tree.
> > Then all architectures could benefit from it. �Having the defconfig
> > files contain only those options which are different from the defaults
> > is certainly more readable, even on x86.
>
> Quite possible. But maintainers would need to be on the lookout of
> people actually using the script, and refusing to apply patches that
> re-introduce the whole big thing.
>
> I also haven't actually seen the script - I don't think Uwe ever
> posted it - so maybe it's some very fragile thing. Hard to judge on no
> real information ;^p
See attachment. It's just:

for each line:
kick $line if $(make ..._defconfig) results in the same config without $line

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K�nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |
From: Russell King - ARM Linux on 12 Jul 2010 16:10
On Mon, Jul 12, 2010 at 10:40:36AM -0700, Linus Torvalds wrote:
> On Mon, Jul 12, 2010 at 10:32 AM, Russell King - ARM Linux
> <linux(a)arm.linux.org.uk> wrote:
> >
> > When you brought up the problem you seemed absolutely convinced
> > that nothing except your solution was going to be acceptable.
>
> That's not true. What's true is that you didn't seem to _understand_
> my solution, so I tried to push the understanding of it.

That's your point of view.

My viewpoint was that I had read your email, thought of some alternative
solution, proposed it and the result was shot down without any apparant
thought about it. That gave the impression that you _only_ wanted to
see your own solution.

The result of that has been very little in the way of progress towards
either your, or my alternative solutions - and apart from a few Kconfig
corner quirk patches, the only major work that's happened has been from
Uwe.

So in that regard, I support Uwe's patches as a means to prevent the
impending loss of build coverage from facilities such as linux-next and
the ARM kernel autobuilder, as that's the only option that's currently
available.

As far as timing goes, I see no reason for it to be merged during -rc.
As has already been said, I'm intending _not_ to make the defconfig
situation any worse by accepting patches which add to the mess, as I'm
also mindful that its exactly those kinds of patches are the cause of
this problem.

Now, I'm happy to take Uwe's tree through mine for the next merge
window _iff_ you find his solution acceptable, and you want that to
happen.

I'll also use this opportunity to warn you now - especially as you also
complained about the amount of churn. There is an effort to try to
allow a single ARM kernel to boot on a wider range of platforms with
more differences than it currently does. This is going to mean a
certain amount of restructuring, and a lot of additional patches over
time to gradually convert the code. So, I'm expecting that there will
be even _more_ patches - some fairly big - to come over the next year
or so.

To give an idea - if we change the structure which defines a machine's
properties, we're going to be changing 348 files under arch/arm to match.
Clearly that's also going to give you a problem with diffstat being
swamped.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Nicolas Pitre on 12 Jul 2010 16:20
On Mon, 12 Jul 2010, Linus Torvalds wrote:

> On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico(a)fluxnic.net> wrote:
> >>
> >> � �Put another way: I realize that fairly late in the -rc series is
> >> actually a really good time to do this, rather than during the merge
> >> window itself when things are in flux. However, while it would be a
> >> good time to pull this for that reason, it's also a _horrible_ time to
> >> pull if it then regresses the defconfig uses, or if it causes horrible
> >> problems for linux-next merging etc.
> >
> > This cannot be any worse than wholesale removal of those files as you
> > were contemplating at some point. �Furthermore, on ARM we have someone
> > providing automatic rebuild of all defconfigs already, so any serious
> > issue should be noticed right away.
>
> You aren't really answering the question. The thing is, the wholesale
> removal wouldn't happen during the late -rc anyway, and it wouldn't
> cause any merge issues (merge conflicts where the file is just to be
> removed are trivial to take care of).

I'll answer for myself only. Others are free to pitch in if they have a
different opinion.

Since this issue came up, RMK refused to merge any changes to the
arch/arm/configs directory. Therefore a lot of those files aren't quite
up to date anymore anyway. We simply skipped the usual defconfig
updates that used to be sent around -rc4. And that's for the few
defconfig files that people cared to update. A bunch of less frequently
used targets are probably out of date since many kernel versions ago.

Those files are mainly used as a convenience for build testing. We tend
to cram as many profiles together as we can to limit the number of
different test builds. The remaining files are (supposed to be)
incompatible configurations. So I doubt anyone is using them verbatim
for deployed systems. If anything they should be reference
configurations not final product ones.

> So I'm really asking about the issue of "how does this work during the
> late -rc phase". Where the _timing_ is the important thing.

Given what I said above, I think that:

1) Those files aren't critical. They're damn useful indeed, but a
glitch in a defconfig file is far from being as important as a glitch
in the very code they refer to. So I don't think this is all that
critical if the pull is applied late in the -rc period.

2) Doing it sooner rather than later is IMHO the best thing to do. At
least we could now focus on maintaining and even improving on that
state rather than going on in circles wondering what to do with it.
People would be able to think about how to update their defconfig
files in the new form now instead of simply not updating it at all as
it has been the case for a while until something happens.

So to me this is all in favor for a merge before next merge window.
During the merge window this would create even more headaches.

> So when I ask about timing and "how does this work in late -rc", it's
> really about how safe it is to do now. Because if it isn't very
> obviously safe, I can't pull it.

As I said above, those files aren't that critical and no one should end
up in trouble if something is not exactly right after this merge. So
this makes it pretty safe to me.

> One part of the "obviously safe" thing is verification for each
> defconfig file that the end result is identical with the reduced
> version. Uwe implied that it was, and that was clearly the intention,
> but was there some explicit verification that yes, the
> before-and-after configurations are 100% the same?

I'll let Uwe answer this.

> Also, I assume that while it's _mostly_ a transparent change to users,
> it does require that people do "make oldconfig" with the reduced
> defconfig file first, in order to avoid having to answer with the
> defaults. No? And the reduced defconfig files obviously do depend on
> the default in the Kconfig files themselves, so it does have that kind
> of fairly subtle semantic change that may not be entirely obvious or
> easy to notice.

That is going to be the case regardless of the merge timing for this.

> So from a timing standpoint, I just want to be convinced that "late
> -rc" really is the right thing. I'm not entirely sure it is, even
> though I do see advantages.

I do too. At least this is positive progress for some bad issue that no
one could ever get very passionate about. Better keep the momentum.

> > I think Uwe could provide his script and add it to the kernel tree.
> > Then all architectures could benefit from it. �Having the defconfig
> > files contain only those options which are different from the defaults
> > is certainly more readable, even on x86.
>
> Quite possible. But maintainers would need to be on the lookout of
> people actually using the script, and refusing to apply patches that
> re-introduce the whole big thing.

Pretty easy to see on the diffstat graph. Anyway, I'm sure once the
script is available then an army of kernel janitors will be busy trying
to find any transgressor.

> I also haven't actually seen the script - I don't think Uwe ever
> posted it - so maybe it's some very fragile thing. Hard to judge on no
> real information ;^p

I'm sure the script was quickly cobbled together as a proof of concept.
But once this defconfig reduction goes in then interest for a solid
script should raise significantly.



Nicolas
From: Nicolas Pitre on 12 Jul 2010 16:30
On Mon, 12 Jul 2010, Russell King - ARM Linux wrote:

> On Mon, Jul 12, 2010 at 10:40:36AM -0700, Linus Torvalds wrote:
> > On Mon, Jul 12, 2010 at 10:32 AM, Russell King - ARM Linux
> > <linux(a)arm.linux.org.uk> wrote:
> > >
> > > When you brought up the problem you seemed absolutely convinced
> > > that nothing except your solution was going to be acceptable.
> >
> > That's not true. What's true is that you didn't seem to _understand_
> > my solution, so I tried to push the understanding of it.
>
> That's your point of view.
>
> My viewpoint was that I had read your email, thought of some alternative
> solution, proposed it and the result was shot down without any apparant
> thought about it. That gave the impression that you _only_ wanted to
> see your own solution.

OK, please let's put this appearance of misunderstanding behind.
The rehashing of it from either parties doesn't produce any good.

> The result of that has been very little in the way of progress towards
> either your, or my alternative solutions - and apart from a few Kconfig
> corner quirk patches, the only major work that's happened has been from
> Uwe.

For the record, I do support Uwe's work too. I do wish it could go in
now so that from that point going forward we could only focus on
improving the thing instead of having to care about implications during
the merge window.

But I do prefer RMK's proposal in the long run. Not only is it more
expressive and clear, but it is easier to maintain going forward too.
But that transition cannot be automated and I doubt the majority of
targets will be converted anytime soon if at all. So at least Uwe's
reduction is quite a good compromise for those.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5
Prev: [RFC] Tight check of pfn_valid on sparsemem
Next: sparc: move is_root_node from private header to of.h