Prev: INDEX build failed for 6.x
Next: INDEX now builds successfully on 6.x
From: Jan Henrik Sylvester on 1 Jun 2010 04:54
The general argument follows below, I just need some anecdotal evidence
first:

Yesterday, I was chasing libintl.so.8, rebuilding all ports that got
bumped, checking with libchk for other libintl.so.8 dependencies, and
forcing a rebuild of all these packages: All but two of them had an
indirect dependency on devel/gettext (and I did email the maintainers of
devel/ccrtp and textproc/gsed linking without a dependency).

Then I did a complete scan for libintl.so.8 and found one file linking
libintl.so.8 that was outside the path that libchk scans (
share/examples/telepathy-qt4/call/.libs/call from net-im/telepathy-qt4).
Yes, 'portupgrade -fr' would have been better as suggested in UPDATING.

So far, no surprise.

Now, many ports got bumped adding a direct dependency that already have
an indirect one. These got rebuild yesterday, either by checking with
libchk or by following UPDATING and rebuilding all (indirect)
dependencies, and now have to be rebuild, again. To me, the new bump
seems to be a waste, since AFAIU indirect and direct dependencies are
not listed in the package any different.

If bumping portrevisions would be done consequently -- for example by
the build cluster recording libdependencies somehow that can be used as
a basis for bumps -- it would help updating, but currently, bumping some
portrevisions does not seem to help and only causes rebuilds being
wasted as in this case.

Moreover, there does not seem to be an agreement, if direct
libdependencies should be listed with indirect already existing. Several
just got them added for gettext, but when I asked for one to be
introduced on another occasion, it got removed shortly afterward again
with pav@ explaining to me that they are not desirable: "We should not
explicitly state any indirect dependencies of any kinds. For the sake of
simplicity." This was the commit:
http://lists.freebsd.org/pipermail/cvs-all/2009-May/288801.html -- I do
not see a difference to the current case.

I think that libdependencies should be recorded for making portrevision
bumps consistent -- either by trying to introduce direct
libdenpendencies for every port that links a shared library or by
recording real shared library dependencies externally (maybe by the
build cluster).

Currently, sometimes all direct and indirect portrevisions are bumped,
sometimes only the direct ones, and sometimes none at all -- in the
second and third case with instructions in UPDATING to rebuild
recursively. For one of my machines, I can use libchk to occasionally
check if something is left behind, but for all of the machines I
transfer packages to, I keep recording lists of packages that need a
forced rebuild. I somehow feel that that should be an automated task
(done by the ports framework itself).

Strictly following UPDATING seems to me not always to be save, either:
If I waited a few weeks and I am told to do two recursive rebuilds (that
are not forced), the second one could miss some portrevision bumps,
because the first one already produced up-to-date ports. If the second
one was due to a shared library bump, I would be in an inconsistent
state. Sometimes, I really wonder about the order in which to follow
instructions in UPDATING -- and I am pretty sure that there is not
always an order that gives a save result: the example I just gave with
several portrevision bumps due to shared library bumps not depending on
each other but with common ports dependent on them cannot be solved, I
guess. Then tools like libchk (or bsdadminscripts) are the only help,
but since they cannot find all libdependencies (dlopen), they are not an
universal solution, either (but seem to have a high rate of success).

My first argument is for portrevision bumps, my second one seems to be
against them. Both approaches seem to have trade-offs, I wish one would
be followed consistently. I know that the decisions are made on a
case-by-case basis, but for my taste, it is too much case-by-case.

Cheers,
Jan Henrik
_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"

From: Garrett Cooper on 1 Jun 2010 12:03
On Tue, Jun 1, 2010 at 1:54 AM, Jan Henrik Sylvester <me(a)janh.de> wrote:
> The general argument follows below, I just need some anecdotal evidence
> first:
>
> Yesterday, I was chasing libintl.so.8, rebuilding all ports that got bumped,
> checking with libchk for other libintl.so.8 dependencies, and forcing a
> rebuild of all these packages: All but two of them had an indirect
> dependency on devel/gettext (and I did email the maintainers of devel/ccrtp
> and textproc/gsed linking without a dependency).
>
> Then I did a complete scan for libintl.so.8 and found one file linking
> libintl.so.8 that was outside the path that libchk scans (
> share/examples/telepathy-qt4/call/.libs/call from net-im/telepathy-qt4).
> Yes, 'portupgrade -fr' would have been better as suggested in UPDATING.
>
> So far, no surprise.
>
> Now, many ports got bumped adding a direct dependency that already have an
> indirect one. These got rebuild yesterday, either by checking with libchk or
> by following UPDATING and rebuilding all (indirect) dependencies, and now
> have to be rebuild, again. To me, the new bump seems to be a waste, since
> AFAIU indirect and direct dependencies are not listed in the package any
> different.
>
> If bumping portrevisions would be done consequently -- for example by the
> build cluster recording libdependencies somehow that can be used as a basis
> for bumps -- it would help updating, but currently, bumping some
> portrevisions does not seem to help and only causes rebuilds being wasted as
> in this case.
>
> Moreover, there does not seem to be an agreement, if direct libdependencies
> should be listed with indirect already existing. Several just got them added
> for gettext, but when I asked for one to be introduced on another occasion,
> it got removed shortly afterward again with pav@ explaining to me that they
> are not desirable: "We should not explicitly state any indirect dependencies
> of any kinds. For the sake of simplicity." This was the commit:
> http://lists.freebsd.org/pipermail/cvs-all/2009-May/288801.html -- I do not
> see a difference to the current case.
>
> I think that libdependencies should be recorded for making portrevision
> bumps consistent -- either by trying to introduce direct libdenpendencies
> for every port that links a shared library or by recording real shared
> library dependencies externally (maybe by the build cluster).
>
> Currently, sometimes all direct and indirect portrevisions are bumped,
> sometimes only the direct ones, and sometimes none at all -- in the second
> and third case with instructions in UPDATING to rebuild recursively. For one
> of my machines, I can use libchk to occasionally check if something is left
> behind, but for all of the machines I transfer packages to, I keep recording
> lists of packages that need a forced rebuild. I somehow feel that that
> should be an automated task (done by the ports framework itself).
>
> Strictly following UPDATING seems to me not always to be save, either: If I
> waited a few weeks and I am told to do two recursive rebuilds (that are not
> forced), the second one could miss some portrevision bumps, because the
> first one already produced up-to-date ports. If the second one was due to a
> shared library bump, I would be in an inconsistent state. Sometimes, I
> really wonder about the order in which to follow instructions in UPDATING --
> and I am pretty sure that there is not always an order that gives a save
> result: the example I just gave with several portrevision bumps due to
> shared library bumps not depending on each other but with common ports
> dependent on them cannot be solved, I guess. Then tools like libchk (or
> bsdadminscripts) are the only help, but since they cannot find all
> libdependencies (dlopen), they are not an universal solution, either (but
> seem to have a high rate of success).
>
> My first argument is for portrevision bumps, my second one seems to be
> against them. Both approaches seem to have trade-offs, I wish one would be
> followed consistently. I know that the decisions are made on a case-by-case
> basis, but for my taste, it is too much case-by-case.

I think that devel/gettext is an unfortunate exception to the rule as
it infects everything under the sun with its `international catalog
support'.

I haven't done portmaster -af in a long time, but unfortunately some
things aren't working as expected (gthumb segfaults on certain
directories), so here we go...

-Garrett
_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"

From: Michel Talon on 2 Jun 2010 04:36
Garrett Cooper wrote:

> > followed consistently. I know that the decisions are made on a
> > case-by-case
> > basis, but for my taste, it is too much case-by-case.
>
> I haven't done portmaster -af in a long time, but unfortunately some
> things aren't working as expected (gthumb segfaults on certain
> directories), so here we go...

In fact these remarks combined show that there are fundamental problems
in the port system, a thing which has been remarked since a long time,
but is regularly denied by many people. Basically it is not in a shape
to be reliably maintained by automatic procedures, contrary to some
concurrence. How to solve the problem, i don't know.



--

Michel TALON

_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"

From: Garrett Cooper on 2 Jun 2010 06:45
On Wed, Jun 2, 2010 at 1:36 AM, Michel Talon <talon(a)lpthe.jussieu.fr> wrote:
> Garrett Cooper wrote:
>
>> > followed consistently. I know that the decisions are made on a
>> > case-by-case
>> > basis, but for my taste, it is too much case-by-case.
>>
>> I haven't done portmaster -af in a long time, but unfortunately some
>> things aren't working as expected (gthumb segfaults on certain
>> directories), so here we go...
>
> In fact these remarks combined show that there are fundamental problems
> in the port system, a thing which has been remarked since a long time,
> but is regularly denied by many people. Basically it is not in a shape
> to be reliably maintained by automatic procedures, contrary to some
> concurrence. How to solve the problem, i don't know.

The lack of reverse dependencies in the pkg_install metadata and
the fact that pkg_install falls back to ports and INDEX (which is
produced by ports) for pkg_version is partly to blame.
The cruxt of the majority of the issues is with the larger package
groups, and the fact that there are some implicit reverse dependencies
that aren't properly resolved because they aren't present in ports.
There are a handful people looking into pkg_install right now
(sort of similar to the way that sysinstall is being treated right
now); it's my goal to take a crack at fixing this problem when I get
the archive(5) pieces in and pkg_install locking work as well. Other
folks are working at fixing how everything is componentized and fits
together in the big picture.
So in summary... there will be a lot of good work coming out of
this area as well, so hopefully these next couple of FreeBSD releases
will be much easier to maintain than the past couple have been from a
packaging perspective.
Thanks,
-Garrett
_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"

From: Peter Jeremy on 3 Jun 2010 08:37
On 2010-Jun-01 10:54:02 +0200, Jan Henrik Sylvester <me(a)janh.de> wrote:
>Yesterday, I was chasing libintl.so.8, rebuilding all ports that got
>bumped, checking with libchk for other libintl.so.8 dependencies, and
>forcing a rebuild of all these packages: All but two of them had an
>indirect dependency on devel/gettext (and I did email the maintainers of
>devel/ccrtp and textproc/gsed linking without a dependency).

This might be unrealistic but, IMHO, these "indirect dependencies" should
not exist. IMHO, there should only be two situations:

1) Port X directly links against or dlopen's libY.so from port Y.

In this case, port Y should be listed in LIB_DEPENDS or equivalent
for port X and port X will need a portrevision bump and rebuild if
the port Y ABI changes (eg a .so version bump)

2) Port X directly links against or dlopen's libZ.so and libZ.so pulls
in libY.so from port Y.

In this case, port X should not be directly accessing any symbols in
libY.so. If the libY.so ABI changes, libZ.so will need to be rebuilt
but unless the libZ.so ABI changes, there should be no need to rebuild
port X.

Are there any other situations that have to be considered?

--
Peter Jeremy
 |  Next  |  Last
Pages: 1 2 3 4
Prev: INDEX build failed for 6.x
Next: INDEX now builds successfully on 6.x