From: Alexander Leidinger on
On Fri, 4 Jun 2010 09:20:48 +0000 "b. f." <bf1783(a)googlemail.com> wrote:

> Alexander Leidinger wrote:
> >Quoting Doug Barton <dougb at FreeBSD.org> (from Thu, 03 Jun 2010
> 11:29:01 -0700):
> >
> >> On 06/03/10 05:39, Matthias Andree wrote:
> >>> Am 03.06.2010 13:30, schrieb Andrey Chernov:
> >>>
> >>>> security/libksba
> >>>> security/libgcrypt
> >>>> (they use libgpg-error)
> >>
> >> So libgpg-error needs to be bumped, but why do things that don't
> >> like directly with gettext need it? One of the major benefits of
> >> shared libraries is to avoid pointless recompiling.
> >
> >The reason (for those interested) is explained here:
> http://www.leidinger.net/blog/2010/06/03/direct-indirect-and-explicit-dependencies-in-progamsports/
>
> Just for the record, the useful
> ports/Tools/scripts/explicit_lib_depends.sh, described and used in
> your link above, may _not_ find libraries that:
>
> -- are needed, but were intended to be statically linked;

Correct. If you are of a way how to detect it after the fact, feel free
to share the info here.

> -- are needed, but loaded via dlopen(3) and friends (this is noted in
> a comment in ports/Tools/scripts/neededlibs.sh );

Correct.

The goal for the scripts are to find entries for LIB_DEPENDS. A static
lib is a BUILD_DEPEND, not a LIB_DEPEND, and something which is opened
via dlopen() is typically not a LIB_DEPEND but a RUN_DEPEND (it is the
other way around and the stuff which is dlopen()en depends upon what
is dlopen()ing it). Additionally if a something is using dlopen(), it
should give sensible error messages when it does not work, so instead
of a cryptic error message which not everyone understands, there should
be (yes, I know, this is an idealistic POV) an error message even your
mother can understand.

I have to tell that those scripts are far from finished, the heuristic
of files to look at needs to be improved, and the translation into
ports-framework variable (like GNOME and X11 ones) needs to be written.
I didn't continue with this back when I was committing those scripts,
as the 3rd party software was far away from a state which would made it
really useful (indirect dependencies show up in runs of the scripts,
but they are not really welcome in our ports).

> -- are needed, and dynamically linked in the usual way, but are not
> referenced in any ELF DT_NEEDED tags. These tags are optional, not
> mandatory, in the System V ABI, and they can be missing for a number
> of reasons. They may not be present in a pre-compiled binary. Or,
> for example. because some ports make shared libraries by converting
> static archives into shared libraries with the linker, the tags can
> sometimes be missing for those libraries. Also, some ports use a
> version of gcc4* wired to devel/binutils, but then directly invoke
> some portion of the older base system binutils. I've seen this lead
> to missing tags in the past.

Apart from the fact that I did not know that, do you know which ports in
the FreeBSD ports collection show this problem? Is there another way to
find this info in then?

Bye,
Alexander.
_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"

From: "b. f." on
On 6/5/10, Alexander Leidinger <Alexander(a)leidinger.net> wrote:
....

>>
>> Just for the record, the useful
>> ports/Tools/scripts/explicit_lib_depends.sh, described and used in
>> your link above, may _not_ find libraries that:
>>
>> -- are needed, but were intended to be statically linked;
>
> Correct. If you are of a way how to detect it after the fact, feel free
> to share the info here.

I don't think there is a good way to do this, but you could perhaps
provide a warning about undefined symbols.

>
>> -- are needed, but loaded via dlopen(3) and friends (this is noted in
>> a comment in ports/Tools/scripts/neededlibs.sh );
>
> Correct.
>
> The goal for the scripts are to find entries for LIB_DEPENDS. A static
> lib is a BUILD_DEPEND, not a LIB_DEPEND, and something which is opened

I take your point. I was just mentioning possible problems with
reference to the OP's use of your script to determine which ports to
rebuild during an update. In that case, one may want to rebuild a
dependent port after updating a static library, even if the dependent
port may not be broken after the update.


> via dlopen() is typically not a LIB_DEPEND but a RUN_DEPEND (it is the
> other way around and the stuff which is dlopen()en depends upon what
> is dlopen()ing it). Additionally if a something is using dlopen(), it

I don't agree with this. Dependent ports often need to test for the
presence of the libraries they intend to dlopen() during configuration
or regression testing, or use headers from the ports that the
libraries belong to during compilation.


> should give sensible error messages when it does not work, so instead
> of a cryptic error message which not everyone understands, there should
> be (yes, I know, this is an idealistic POV) an error message even your
> mother can understand.

Well, let us say the average user. You don't know my mother. :)


> I have to tell that those scripts are far from finished, the heuristic
> of files to look at needs to be improved, and the translation into
> ports-framework variable (like GNOME and X11 ones) needs to be written.
> I didn't continue with this back when I was committing those scripts,
> as the 3rd party software was far away from a state which would made it
> really useful (indirect dependencies show up in runs of the scripts,
> but they are not really welcome in our ports).
>

Yes, I didn't expect them to be highly polished. But they are already
more elaborate than the rough, ad hoc methods I often use, anyway.


>> -- are needed, and dynamically linked in the usual way, but are not
>> referenced in any ELF DT_NEEDED tags. These tags are optional, not
>> mandatory, in the System V ABI, and they can be missing for a number
>> of reasons. They may not be present in a pre-compiled binary. Or,
>> for example. because some ports make shared libraries by converting
>> static archives into shared libraries with the linker, the tags can
>> sometimes be missing for those libraries. Also, some ports use a
>> version of gcc4* wired to devel/binutils, but then directly invoke
>> some portion of the older base system binutils. I've seen this lead
>> to missing tags in the past.
>
> Apart from the fact that I did not know that, do you know which ports in
> the FreeBSD ports collection show this problem? Is there another way to
> find this info in then?

Well, these are admittedly uncommon cases, but I just thought I'd
mention them in case someone using your script to determine which
ports to update encountered one of them. With respect to the
conversion of static libraries into shared libraries, anything that is
using the '--whole-archive' flag during linking bears further
examination. (It doesn't necessarily mean that the DT_NEEDED tags are
missing, just that they _may_ be missing.) Among the ports that do
this are older ports (often Fortran-related), or ports that once used
static libraries because they were slightly more efficient, or more
secure, but have since been modified to also build shared libraries in
a very simple way: graphics/f90gl, math/arpack, math/atlas, math/blas,
math/lapack, math/lapack95, math/metis, math/scalapack, math/spooles,
math/suitesparse, etc. You can see the methods they use in the
respective port Makefiles. Where needed, they may be fixed to include
the proper tags, and I submitted patches to do this for some of them,
but they were never committed, making linking for some of these
libraries a real pain. See, for example, the unnecessarily complex
(and wrong) description of how to link with the math/atlas _shared_
libraries in math/atlas/pkg-descr.

As for ports that mix tool-chains, this includes a great many of the
ports that USE_GCC=4.4+ or USE_FORTRAN, because, even though the gcc
maintainer (finally) fixed the problem with the auto-detection of
devel/binutils that could produce unpredictable results with respect
to which tool-chain was used by the compiler in recent gcc snapshots,
he did not also instruct those ports (and their dependencies) to use
devel/binutils when invoking the tool-chain utilities _directly_, so
they can (and do) mix tool-chains during some port builds. For
example, the ports I mentioned above use lang/gcc44, which indirectly
invokes the assembler and linker from devel/binutils, but then invoke
the older base system linker directly in the port Makefiles. Apart
from other problems that this can cause, I once noticed that this
resulted in a loss of tags in the resulting binaries. The fact that
such a loss can occur is obviously a bug, so I don't claim that this
always happens, or that it happens with the current default setup.

b.
_______________________________________________
freebsd-ports(a)freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"