crossdev and multilib interference

classic Classic list List threaded Threaded
85 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

crossdev and multilib interference

hasufell
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.

crossdev for example sets in their pkg-config wrappers:

PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"

Now, SYSROOT is chosen from multiple conditions. When emerging a
package, that happens to be "/" and thus results in:
  "//usr/lib/pkgconfig://usr/share/pkgconfig"

Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...

This is not a problem most of the time if the package just wants to
get the libs to link against.

However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.

That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)

and there are probably more.

A simple workaround is:
PKG_CONFIG="pkg-config" emerge foo

But I think that is not appropriate to set in the eclass. How can we
solve this? Don't bikeshed.
-----BEGIN PGP SIGNATURE-----

iQJ8BAEBCgBmBQJTIIE5XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQzMDlCNDQ4NjEyNDI4NjA5REVEMDI3MzIy
MjBDRDFDNUJERUVEMDIwAAoJECIM0cW97tAgLvMQALJT5uZFBTL5mHNBjezudMHo
ceHY3TzLfeIkWHedCLU9TAapfLjl677W0jbg25zkLayPtUR3voEAIm6xFZtF2CAS
VsBpArieXQWqtxSrT9hHC1dJeAWQvQs1kyKXBb5ido8sEBb7WpHFlEqwmUY5bn33
TZjGjcQ68EojbcFQl0vmJRx1/bgXxOr9Ir4Y1bFX92S9ENhERnisu3zZrcvC+PyH
XqXg+wFoJfxu1fSL4/llRDfyr0UJWlWdMeCpvwgkhCpn66CBwc50BwokMP4f4x3G
YDbi+1Ep4GIBVHLd5MlfZOkeqhzPQRD10lbnOHmW7Wuh6Mu87UI7G9xHWZcNyEkS
U9Ny0ejyqnx0j5TMgMP/IcpBR1PaRcceTLFYhwJrYucgcB6/YZ1sP81Yce7f2Riy
M6OZontsv8GVbPA4tsgsV4wob6XUzn6d47p4jwbq67u3GUX6QU7fNB0yJ2ga56qV
xvIaEgdOFsAZHOyix6zfTa2AqpE2LQiVfwEF2pI4J4bTCfy7DvfWhuvA5IwWyyFA
jPiGr5xEns85v2q+dagUo/iup9gzbgGQs+dH6w3wXTkip72lnbxHwiz8Pa2eIXVl
+Tvo9vcdVSzw68tF30sS005+HNorCkj/pqdC7FND/KCyC7r3aESmagibqKdUhHPc
9sN1RjgyzXYst5mvQ1Hg
=J4Qp
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Alexandre Rostovtsev-2
On Wed, 2014-03-12 at 15:46 +0000, hasufell wrote:

> We have a problem where the crossdev pkg-config wrapper scripts
> interfere with multilib.
>
> crossdev for example sets in their pkg-config wrappers:
>
> PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
>
> Now, SYSROOT is chosen from multiple conditions. When emerging a
> package, that happens to be "/" and thus results in:
>   "//usr/lib/pkgconfig://usr/share/pkgconfig"
>
> Build systems like autotools will pick the crossdev provided
> "i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
> override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
> find the pkg-config files in /usr/lib64/...
>
> This is not a problem most of the time if the package just wants to
> get the libs to link against.
>
> However, every package that tries to access variables that are
> different between /usr/lib32/pkgconfig/foo.pc and
> /usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
> unexpected results.
>
> That already happens for
> x11-libs/libva-vdpau-driver
> x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
>
> and there are probably more.
>
> A simple workaround is:
> PKG_CONFIG="pkg-config" emerge foo
>
> But I think that is not appropriate to set in the eclass. How can we
> solve this? Don't bikeshed.
Two possibilities:
1. Don't allow crossdev to handle targets which are natively handled by
multilib profiles. For example, is there any legitimate reason for
wanting crossdev's i686 wrappers when on a multilib amd64 profile?
2. Have crossdev install its wrappers in a prefix, for example
in /usr/libexec/crossdev, which gets added to PATH by cross-emerge.

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Alexis Ballier-2
On Wed, 12 Mar 2014 12:06:32 -0400
Alexandre Rostovtsev <[hidden email]> wrote:

> Two possibilities:
> 1. Don't allow crossdev to handle targets which are natively handled
> by multilib profiles. For example, is there any legitimate reason for
> wanting crossdev's i686 wrappers when on a multilib amd64 profile?

yes, serving as a distcc server for x86 hosts or using 'cross emerge'
to build a x86 root from scratch

> 2. Have crossdev install its wrappers in a prefix, for example
> in /usr/libexec/crossdev, which gets added to PATH by cross-emerge.

lgtm

Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Michał Górny-5
In reply to this post by hasufell
Dnia 2014-03-12, o godz. 15:46:01
hasufell <[hidden email]> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> We have a problem where the crossdev pkg-config wrapper scripts
> interfere with multilib.
>
> crossdev for example sets in their pkg-config wrappers:
>
> PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
>
> Now, SYSROOT is chosen from multiple conditions. When emerging a
> package, that happens to be "/" and thus results in:
>   "//usr/lib/pkgconfig://usr/share/pkgconfig"
>
> Build systems like autotools will pick the crossdev provided
> "i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
> override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
> find the pkg-config files in /usr/lib64/...
>
> This is not a problem most of the time if the package just wants to
> get the libs to link against.
>
> However, every package that tries to access variables that are
> different between /usr/lib32/pkgconfig/foo.pc and
> /usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
> unexpected results.
>
> That already happens for
> x11-libs/libva-vdpau-driver
> x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
>
> and there are probably more.
Another possible workaround is to make pkgconfig true-multilib. Then it
would own i686-pc-linux-gnu-pkgconfig, and that executable would work
correctly. More than that, we could work on killing the PKG_CONFIG_PATH
hack.

--
Best regards,
Michał Górny

signature.asc (985 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Alexandre Rostovtsev-2
On Thu, 2014-03-13 at 09:55 +0100, Michał Górny wrote:

> Dnia 2014-03-12, o godz. 15:46:01
> hasufell <[hidden email]> napisał(a):
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > We have a problem where the crossdev pkg-config wrapper scripts
> > interfere with multilib.
> >
> > crossdev for example sets in their pkg-config wrappers:
> >
> > PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
> >
> > Now, SYSROOT is chosen from multiple conditions. When emerging a
> > package, that happens to be "/" and thus results in:
> >   "//usr/lib/pkgconfig://usr/share/pkgconfig"
> >
> > Build systems like autotools will pick the crossdev provided
> > "i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
> > override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
> > find the pkg-config files in /usr/lib64/...
> >
> > This is not a problem most of the time if the package just wants to
> > get the libs to link against.
> >
> > However, every package that tries to access variables that are
> > different between /usr/lib32/pkgconfig/foo.pc and
> > /usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
> > unexpected results.
> >
> > That already happens for
> > x11-libs/libva-vdpau-driver
> > x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
> >
> > and there are probably more.
>
> Another possible workaround is to make pkgconfig true-multilib. Then it
> would own i686-pc-linux-gnu-pkgconfig, and that executable would work
> correctly. More than that, we could work on killing the PKG_CONFIG_PATH
> hack.
That won't work with current versions of crossdev because they blindly
create/delete their pkg-config wrappers without checking if they are
overwriting or removing something that belongs to another package.

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Greg Turner
In reply to this post by Alexandre Rostovtsev-2
Just a few practical notes on this...

On Wed, Mar 12, 2014 at 9:06 AM, Alexandre Rostovtsev <[hidden email]> wrote:
> Now, SYSROOT is chosen from multiple conditions. When emerging a
> package, that happens to be "/" and thus results in:
>   "//usr/lib/pkgconfig://usr/share/pkgconfig"

Bleh.  This is where I obligatorily remind everyone that // != /. POSIX, cygwin, blahblahblah.  In short, paths matching the regex "^//[^/]" are trouble, and will eventually force me to fix your code once I get back to gentoo-cygwin hacking.
 
> Build systems like autotools will pick the crossdev provided
> "i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
> override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
> find the pkg-config files in /usr/lib64/...

So do Gentoo's own toolchain*.eclasses, breaking many $(tc-getFOO) invocations in non-best multilib-build ABIS for which a matching crossdev target is installed (substituting ${FOO:-$(tc-getFOO)} works-around these problems in every instance I've seen; it probably wouldn't hurt to make such substitutions systematically, when multilib-utizing ebuilds and eclasses).
 
> This is not a problem most of the time if the package just wants to
> get the libs to link against

lots of "ignoring blahlib.so 'cause it's for the wrong ABI"-type warnings are a good sign your ebuild may have fallen into this rabbit-hole.
 
 
> However, every package that tries to access variables that are
> different between /usr/lib32/pkgconfig/foo.pc and
> /usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
> unexpected results.
>
> That already happens for
> x11-libs/libva-vdpau-driver
> x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
>
> and there are probably more.

"Every" might be too strong, but... yeah, tons.

cmake-multilib.eclass, for example, breaks in mind-warpingly subtle and confusing ways on USE="abi_x86_{32,64}" multilib hosts with i686-pc-linux-gnu crossdev installed (when combined with some other issues in that eclass, this results in correct qawarns about ignored ldflags on devel profiles -- an ugly work-around is in my overlay, but I'm not happy with it, so it's been languishing in my rainy-day todo queue.  I'd be thrilled to see a solution to the underlying problem, so I don't feel compelled to salvage the ugly parts).

As for how to fix it, if foo-bar-baz-quux crossdev targets are at ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in ${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that, seems perfectly reasonable... heck, pure speculation, but it might even noticeably speed up day-to-day $PATH searching on systems with lots of crossdev targets installed.
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Greg Turner
In reply to this post by Alexandre Rostovtsev-2

On Wed, Mar 12, 2014 at 9:06 AM, Alexandre Rostovtsev <[hidden email]> wrote:
is there any legitimate reason for
wanting crossdev's i686 wrappers when on a multilib amd64 profile?

One other note: legitimacy is in the eye of the legitimizer, I suppose, but there are sometimes practical reasons to want a no-multilib compiler.

A trivial example is that some primitive build scripts go ape when CC has a space in it (but this is certainly debatable as to its legitimacy, given the obvious work-arounds).
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
In reply to this post by Michał Górny-5
On Thu 13 Mar 2014 09:55:02 Michał Górny wrote:

> Dnia 2014-03-12, o godz. 15:46:01 hasufell napisał(a):
> > We have a problem where the crossdev pkg-config wrapper scripts
> > interfere with multilib.
> >
> > crossdev for example sets in their pkg-config wrappers:
> >
> > PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgco
> > nfig"
> >
> > Now, SYSROOT is chosen from multiple conditions. When emerging a
> >
> > package, that happens to be "/" and thus results in:
> >   "//usr/lib/pkgconfig://usr/share/pkgconfig"
this behavior is more bug than feature.  the preference is to utilize SYSROOT
in src_* functions first and ROOT in pkg_* functions.  i'll have to see if
EBUILD_PHASE is exported to shell scripts ...

> > Build systems like autotools will pick the crossdev provided
> > "i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
> > override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
> > find the pkg-config files in /usr/lib64/...
> >
> > This is not a problem most of the time if the package just wants to
> > get the libs to link against.
> >
> > However, every package that tries to access variables that are
> > different between /usr/lib32/pkgconfig/foo.pc and
> > /usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
> > unexpected results.
> >
> > That already happens for
> > x11-libs/libva-vdpau-driver
> > x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
> >
> > and there are probably more.
there have been reports dating back even longer.  e.g. people used crossdev to
create i686-pc-linux-gnu and then tried to build sandbox.  it's just the
number of people hitting this was fairly low (like ~1 a year).  and for them,
it was possible to just use a diff tuple for their i686 compiler.

> Another possible workaround is to make pkgconfig true-multilib. Then it
> would own i686-pc-linux-gnu-pkgconfig, and that executable would work
> correctly. More than that, we could work on killing the PKG_CONFIG_PATH
> hack.

that by itself doesn't really help.  the wrappers are designed to be used with
the respective cross-compiler (SYSROOT by default), not default to the host
system.

when you run `crossdev i686-pc-linux-gnu`, it owns that tuple.  that includes
i686-pc-linux-gnu-pkg-config.

if we're going to have the multilib system lie and use a tuple that doesn't
actually exist, we either:

(1) override all the vars so they point back to the real toolchain.
this doesn't scale when you consider helper tools like the legacy sdl-config
or the extended set of tools that binutils/gcc/etc... install.  it's mitigated
by the fact the set of vars in play most of the time is low.

(2) use tuples with loaded vendor fields to reduce the chance of collisions.
e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead of
i686-pc-linux-gnu would defeat any automatic path searches.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
In reply to this post by Greg Turner
On Sun 16 Mar 2014 04:50:33 Greg Turner wrote:
> cmake-multilib.eclass, for example, breaks in mind-warpingly subtle and
> confusing ways on USE="abi_x86_{32,64}" multilib hosts with
> i686-pc-linux-gnu crossdev installed (when combined with some other issues
> in that eclass, this results in correct qawarns about ignored ldflags on
> devel profiles -- an ugly work-around is in my overlay, but I'm not happy
> with it, so it's been languishing in my rainy-day todo queue.  I'd be
> thrilled to see a solution to the underlying problem, so I don't feel
> compelled to salvage the ugly parts).

cmake is completely broken when it comes to library searching and multilib and
cross-compiling.  it will happily look in hardcoded / paths to test for the
existence of files as well as directly execute `pkg-config`.  it's a great
example of people saying "autotools is crap, so let's invent our own kind of
crap and ignore lessons learned".  this isn't the fault of cmake eclasses, but
it'd be nice if we could someone standardize the hacks in there so we don't
have to duplicate across ebuilds.

just look at how cmake internally utilizes CMAKE_FIND_ROOT_PATH.  or grep
"pkg-config" in /usr/share/cmake/.  or look at find_library usage in cmake
macros.  it's fundamentally screwed up right now :(.

> As for how to fix it, if foo-bar-baz-quux crossdev targets are at
> ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
> ${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that, seems
> perfectly reasonable... heck, pure speculation, but it might even
> noticeably speed up day-to-day $PATH searching on systems with lots of
> crossdev targets installed.

if they're in $PATH, then the exact location is irrelevant.  they need not be
in /usr/bin to cause a problem.  if they're not in $PATH, then you're breaking
the cross-compilers and that is unacceptable.

putting CHOST things in /usr/CTARGET is incorrect.  /usr/CHOST/CTARGET/ is for
hosting helper programs specific to CTARGET that run on CHOST.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Steven J. Long
Mike Frysinger wrote:

> Greg Turner wrote:
> > As for how to fix it, if foo-bar-baz-quux crossdev targets are at
> > ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
> > ${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that,
> > seems perfectly reasonable... heck, pure speculation, but it might
> > even noticeably speed up day-to-day $PATH searching on systems with
> > lots of crossdev targets installed.
>
> if they're in $PATH, then the exact location is irrelevant.
> they need not be in /usr/bin to cause a problem.
> if they're not in $PATH, then you're breaking the cross-compilers
> and that is unacceptable.

Cross-compilation should be supported via cross-emerge, and perhaps a small
script the cross-compiler sources to setup the env (ie prefix to PATH in
this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
a straight x86 platform, instead of the normal multilib setup. The
latter being used by the former (I'd have thought it was already done.)

The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point. It's *borked*, plain and
simple, so fix it please or expect people to come up with other solutions
[1]; fragmenting the effort, and making cross-compilers lives harder, as
we try to blend together a working solution from various efforts. The
exact thing crossdev is supposed to answer.

> putting CHOST things in /usr/CTARGET is incorrect.  /usr/CHOST/CTARGET/
> is for hosting helper programs specific to CTARGET that run on CHOST.

Great, make it part of the env script. Though CHOST is usually the "target"
for most packages, so not sure of the relevance apart from toolchain.
Greg appeared to be saying use /usr/CHOST/usr/bin/gcc for example (istr
it's effectively a root under there) or just prefix the relevant bindirs
to PATH, along with whatever other voodoo you need. sed for instance is
still going to come from native CBUILD /bin.

Regards,
steveL

[1] https://github.com/chewi/cross-boss [still in preview]
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
On Wed 26 Mar 2014 12:25:29 Steven J. Long wrote:

> Mike Frysinger wrote:
> > Greg Turner wrote:
> > > As for how to fix it, if foo-bar-baz-quux crossdev targets are at
> > > ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
> > > ${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that,
> > > seems perfectly reasonable... heck, pure speculation, but it might
> > > even noticeably speed up day-to-day $PATH searching on systems with
> > > lots of crossdev targets installed.
> >
> > if they're in $PATH, then the exact location is irrelevant.
> > they need not be in /usr/bin to cause a problem.
> > if they're not in $PATH, then you're breaking the cross-compilers
> > and that is unacceptable.
>
> Cross-compilation should be supported via cross-emerge, and perhaps a small
> script the cross-compiler sources to setup the env (ie prefix to PATH in
> this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
> a straight x86 platform, instead of the normal multilib setup. The
> latter being used by the former (I'd have thought it was already done.)
>
> The cross tools should NOT pollute the default PATH, simply because the
> user happened to run crossdev at some point. It's *borked*, plain and
> simple, so fix it please or expect people to come up with other solutions
> [1]; fragmenting the effort, and making cross-compilers lives harder, as
> we try to blend together a working solution from various efforts. The
> exact thing crossdev is supposed to answer.
that's bs.  people install crossdev to get a cross-compile environment, not to
get something that only works through `emerge`.  attempting to restrict it so
it only works through `emerge` is unacceptable and it has never been that way.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Ian Stakenvicius-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26/03/14 12:12 PM, Mike Frysinger wrote:

> On Wed 26 Mar 2014 12:25:29 Steven J. Long wrote:
>> Mike Frysinger wrote:
>>> Greg Turner wrote:
>>>> As for how to fix it, if foo-bar-baz-quux crossdev targets
>>>> are at ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
>>>> ${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something
>>>> like that, seems perfectly reasonable... heck, pure
>>>> speculation, but it might even noticeably speed up day-to-day
>>>> $PATH searching on systems with lots of crossdev targets
>>>> installed.
>>>
>>> if they're in $PATH, then the exact location is irrelevant.
>>> they need not be in /usr/bin to cause a problem. if they're not
>>> in $PATH, then you're breaking the cross-compilers and that is
>>> unacceptable.
>>
>> Cross-compilation should be supported via cross-emerge, and
>> perhaps a small script the cross-compiler sources to setup the
>> env (ie prefix to PATH in this case) for using CHOST-* tools,
>> like x86-pc-linux-gnu-gcc targetting a straight x86 platform,
>> instead of the normal multilib setup. The latter being used by
>> the former (I'd have thought it was already done.)
>>
>> The cross tools should NOT pollute the default PATH, simply
>> because the user happened to run crossdev at some point. It's
>> *borked*, plain and simple, so fix it please or expect people to
>> come up with other solutions [1]; fragmenting the effort, and
>> making cross-compilers lives harder, as we try to blend together
>> a working solution from various efforts. The exact thing crossdev
>> is supposed to answer.
>
> that's bs.  people install crossdev to get a cross-compile
> environment, not to get something that only works through `emerge`.
> attempting to restrict it so it only works through `emerge` is
> unacceptable and it has never been that way. -mike
>

it -does- make sense though to limit anything that one wants to EMERGE
with the crossdev, to require the use of cross-emerge.  Would it not
be possible to somehow ensure the crossdev tools are ignored
in/removed from/cannot pollute the standard emerge environment?  Are
there any use cases where one -would- want the crossdev to be used in
a standard emerge environment instead of using cross-emerge ?




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlMy/xkACgkQ2ugaI38ACPClAAD/bwSIWjCu32eDlf3faqnqhvc3
94JxKfSwY3pPv7X4A68A/1g8KSov5e/BHGYXyhlyCd8j3Bc+IukxoNYMXiXiluh7
=TMGw
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
On Wed 26 Mar 2014 12:23:53 Ian Stakenvicius wrote:

> On 26/03/14 12:12 PM, Mike Frysinger wrote:
> > that's bs.  people install crossdev to get a cross-compile
> > environment, not to get something that only works through `emerge`.
> > attempting to restrict it so it only works through `emerge` is
> > unacceptable and it has never been that way.
>
> it -does- make sense though to limit anything that one wants to EMERGE
> with the crossdev, to require the use of cross-emerge.  Would it not
> be possible to somehow ensure the crossdev tools are ignored
> in/removed from/cannot pollute the standard emerge environment?  Are
> there any use cases where one -would- want the crossdev to be used in
> a standard emerge environment instead of using cross-emerge ?
you've lost me.  when you `emerge-$CTARGET`, that package doesn't go anywhere
near your ROOT=/ system.  it's entirely contained in /usr/$CTARGET/.

when you run `crossdev $CTARGET`, it installs all the standard $CTARGET-xxx
tools in /usr/bin.  this isn't "polluting" the environment at all ... in fact,
they're living right alongside existing tools.

as i pointed out elsewhere in this thread, the problem is that multilib relies
on automatic detection of the toolchain *failing* so that it falls back to the
native value.  in other words, when you run `./configure --host=i686-pc-linux-
gnu`, it tries to find e.g. i686-pc-linux-gnu-ar.  it doesn't exist so the
fallback is used (plain `ar`).  multilib is using these tuples so that the
standard checks (autoconf/eclasses/etc...) trigger in the right ways for the
cpu/os/userland combinations.

since crossdev installs a full proper toolchain for the target, the one
multilib was using to lie now exists and its toolchain is used instead.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Alexandre Rostovtsev-2
On Wed, 2014-03-26 at 22:41 -0400, Mike Frysinger wrote:

> On Wed 26 Mar 2014 12:23:53 Ian Stakenvicius wrote:
> > On 26/03/14 12:12 PM, Mike Frysinger wrote:
> > > that's bs.  people install crossdev to get a cross-compile
> > > environment, not to get something that only works through `emerge`.
> > > attempting to restrict it so it only works through `emerge` is
> > > unacceptable and it has never been that way.
> >
> > it -does- make sense though to limit anything that one wants to EMERGE
> > with the crossdev, to require the use of cross-emerge.  Would it not
> > be possible to somehow ensure the crossdev tools are ignored
> > in/removed from/cannot pollute the standard emerge environment?  Are
> > there any use cases where one -would- want the crossdev to be used in
> > a standard emerge environment instead of using cross-emerge ?
>
> you've lost me.  when you `emerge-$CTARGET`, that package doesn't go anywhere
> near your ROOT=/ system.  it's entirely contained in /usr/$CTARGET/.
>
> when you run `crossdev $CTARGET`, it installs all the standard $CTARGET-xxx
> tools in /usr/bin.  this isn't "polluting" the environment at all ... in fact,
> they're living right alongside existing tools.
>
> as i pointed out elsewhere in this thread, the problem is that multilib relies
> on automatic detection of the toolchain *failing* so that it falls back to the
> native value.  in other words, when you run `./configure --host=i686-pc-linux-
> gnu`, it tries to find e.g. i686-pc-linux-gnu-ar.  it doesn't exist so the
> fallback is used (plain `ar`).  multilib is using these tuples so that the
> standard checks (autoconf/eclasses/etc...) trigger in the right ways for the
> cpu/os/userland combinations.
>
> since crossdev installs a full proper toolchain for the target, the one
> multilib was using to lie now exists and its toolchain is used instead.
> -mike
Crossdev is "polluting the environment" in the specific case that we are
talking about - secondary native ABIs on a multilib system.

An amd64 multilib system is not expected to build MIPS binaries that
would be hosted on itself. So of course anyone using amd64 undersands
that mips-pc-linux-gnu-ar is part of a cross-compile toolchain, no
matter whether it's in /usr/bin or /usr/libexec/crossdev or anywhere in
the filesystem.

However, an i686 crossdev on an amd64 multilib system is a fundamentally
different situation. An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
On Thu 27 Mar 2014 00:41:47 Alexandre Rostovtsev wrote:

> On Wed, 2014-03-26 at 22:41 -0400, Mike Frysinger wrote:
> > On Wed 26 Mar 2014 12:23:53 Ian Stakenvicius wrote:
> > > On 26/03/14 12:12 PM, Mike Frysinger wrote:
> > > > that's bs.  people install crossdev to get a cross-compile
> > > > environment, not to get something that only works through `emerge`.
> > > > attempting to restrict it so it only works through `emerge` is
> > > > unacceptable and it has never been that way.
> > >
> > > it -does- make sense though to limit anything that one wants to EMERGE
> > > with the crossdev, to require the use of cross-emerge.  Would it not
> > > be possible to somehow ensure the crossdev tools are ignored
> > > in/removed from/cannot pollute the standard emerge environment?  Are
> > > there any use cases where one -would- want the crossdev to be used in
> > > a standard emerge environment instead of using cross-emerge ?
> >
> > you've lost me.  when you `emerge-$CTARGET`, that package doesn't go
> > anywhere near your ROOT=/ system.  it's entirely contained in
> > /usr/$CTARGET/.
> >
> > when you run `crossdev $CTARGET`, it installs all the standard
> > $CTARGET-xxx
> > tools in /usr/bin.  this isn't "polluting" the environment at all ... in
> > fact, they're living right alongside existing tools.
> >
> > as i pointed out elsewhere in this thread, the problem is that multilib
> > relies on automatic detection of the toolchain *failing* so that it falls
> > back to the native value.  in other words, when you run `./configure
> > --host=i686-pc-linux- gnu`, it tries to find e.g. i686-pc-linux-gnu-ar.
> > it doesn't exist so the fallback is used (plain `ar`).  multilib is using
> > these tuples so that the standard checks (autoconf/eclasses/etc...)
> > trigger in the right ways for the cpu/os/userland combinations.
> >
> > since crossdev installs a full proper toolchain for the target, the one
> > multilib was using to lie now exists and its toolchain is used instead.
>
> Crossdev is "polluting the environment" in the specific case that we are
> talking about - secondary native ABIs on a multilib system.
>
> An amd64 multilib system is not expected to build MIPS binaries that
> would be hosted on itself. So of course anyone using amd64 undersands
> that mips-pc-linux-gnu-ar is part of a cross-compile toolchain, no
> matter whether it's in /usr/bin or /usr/libexec/crossdev or anywhere in
> the filesystem.
>
> However, an i686 crossdev on an amd64 multilib system is a fundamentally
> different situation.
no, it is not

> An amd64 multilib system *is* expected to build x86
> binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
> expected to be not a part of any cross-compile toolchain, but a part of
> the native toolchain for the machine's secondary native ABI. Especially
> when i686-pc-linux-gnu-ar is in /usr/bin.

sure, and it works just fine when you use the correct toolchain.  if the user
wants to build an ABI using their default toolchain, they can pass the right
ABI flag for it.

but that's irrelevant to how our multilib implementation picks its fake
CHOST_$ABI to take care of implementing things.  if anything, the current
system is provably broken because the default ends up executing unqualified
`ar` and `ranlib` friends.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
In reply to this post by Mike Frysinger
On Wed 26 Mar 2014 01:17:14 Mike Frysinger wrote:
> (2) use tuples with loaded vendor fields to reduce the chance of collisions.
> e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead
> of i686-pc-linux-gnu would defeat any automatic path searches.

this patch keeps the status quo.  although the status quo is broken, but we
can sort that out independently.
-mike

--- profiles/arch/amd64/make.defaults 18 Jan 2014 01:03:24 -0000 1.21
+++ profiles/arch/amd64/make.defaults 27 Mar 2014 06:13:22 -0000
@@ -21,17 +21,17 @@ ABI="amd64"
 # 64bit specific settings.
 CFLAGS_amd64="-m64"
 LDFLAGS_amd64="-m elf_x86_64"
-CHOST_amd64="x86_64-pc-linux-gnu"
+CHOST_amd64="${CHOST}"
 
 # 32bit specific settings.
 CFLAGS_x86="-m32"
 LDFLAGS_x86="-m elf_i386"
-CHOST_x86="i686-pc-linux-gnu"
+CHOST_x86="i686-gentoo%multilib-linux-gnu"
 
 # 64-32bit specific settings.
 CFLAGS_x32="-mx32"
 LDFLAGS_x32="-m elf32_x86_64"
-CHOST_x32="x86_64-pc-linux-gnux32"
+CHOST_x32="x86_64-gentoo%multilib-linux-gnux32"
 
 # 2006/10/24 - Simon Stelling <[hidden email]>
 # They are masked, but we can enable them anyway for those who have

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Alexandre Rostovtsev-2
In reply to this post by Mike Frysinger
On Thu, 2014-03-27 at 02:07 -0400, Mike Frysinger wrote:
> > An amd64 multilib system *is* expected to build x86
> > binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
> > expected to be not a part of any cross-compile toolchain, but a part of
> > the native toolchain for the machine's secondary native ABI. Especially
> > when i686-pc-linux-gnu-ar is in /usr/bin.
>
> sure, and it works just fine when you use the correct toolchain.  if the user
> wants to build an ABI using their default toolchain, they can pass the right
> ABI flag for it.

They can't pass the right ABI flag because only the core parts of the
toolchain have the concept of an ABI flag.

Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
various libraries? Are you going to patch all of them to respect "-m32"?

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Mike Frysinger
On Thu 27 Mar 2014 02:31:01 Alexandre Rostovtsev wrote:

> On Thu, 2014-03-27 at 02:07 -0400, Mike Frysinger wrote:
> > > An amd64 multilib system *is* expected to build x86
> > > binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
> > > expected to be not a part of any cross-compile toolchain, but a part of
> > > the native toolchain for the machine's secondary native ABI. Especially
> > > when i686-pc-linux-gnu-ar is in /usr/bin.
> >
> > sure, and it works just fine when you use the correct toolchain.  if the
> > user wants to build an ABI using their default toolchain, they can pass
> > the right ABI flag for it.
>
> They can't pass the right ABI flag because only the core parts of the
> toolchain have the concept of an ABI flag.
>
> Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
> clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
> various libraries? Are you going to patch all of them to respect "-m32"?
pkg-config does need fixing in some way.  we already know this.  it's why the
multilib eclasses currently set PKG_CONFIG_XXX vars -- preciously so the
correct ABI dir is utilized.  and this breaks when using some build systems
(like scons) where the env gets blown away (although we also know scons
sucks).

i don't care about the *-config scripts.  that's a dead concept long ago
proven to suck and anything still using it needs fixing.
-mike

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Michał Górny-5
Dnia 2014-03-27, o godz. 02:41:21
Mike Frysinger <[hidden email]> napisał(a):

> On Thu 27 Mar 2014 02:31:01 Alexandre Rostovtsev wrote:
> > On Thu, 2014-03-27 at 02:07 -0400, Mike Frysinger wrote:
> > > > An amd64 multilib system *is* expected to build x86
> > > > binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
> > > > expected to be not a part of any cross-compile toolchain, but a part of
> > > > the native toolchain for the machine's secondary native ABI. Especially
> > > > when i686-pc-linux-gnu-ar is in /usr/bin.
> > >
> > > sure, and it works just fine when you use the correct toolchain.  if the
> > > user wants to build an ABI using their default toolchain, they can pass
> > > the right ABI flag for it.
> >
> > They can't pass the right ABI flag because only the core parts of the
> > toolchain have the concept of an ABI flag.
> >
> > Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
> > clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
> > various libraries? Are you going to patch all of them to respect "-m32"?
>
> pkg-config does need fixing in some way.  we already know this.  it's why the
> multilib eclasses currently set PKG_CONFIG_XXX vars -- preciously so the
> correct ABI dir is utilized.  and this breaks when using some build systems
> (like scons) where the env gets blown away (although we also know scons
> sucks).
>
> i don't care about the *-config scripts.  that's a dead concept long ago
> proven to suck and anything still using it needs fixing.
Because it's everyone else that *always* does something wrong,
and the rising number of work-arounds in the eclass just proves you're
doing it the correct way.

It's sad that decisions are made by man who *does not care* about most
of the real-life things but cares much for his own precious tools that
get priority over eclasses, and whenever they are essentially broken
by design, you say that the world is actually broken and they
are the only fine thing.

--
Best regards,
Michał Górny

signature.asc (985 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: crossdev and multilib interference

Michał Górny-5
In reply to this post by Mike Frysinger
Dnia 2014-03-27, o godz. 02:13:52
Mike Frysinger <[hidden email]> napisał(a):

> On Wed 26 Mar 2014 01:17:14 Mike Frysinger wrote:
> > (2) use tuples with loaded vendor fields to reduce the chance of collisions.
> > e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead
> > of i686-pc-linux-gnu would defeat any automatic path searches.
>
> this patch keeps the status quo.  although the status quo is broken, but we
> can sort that out independently.

Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.

> --- profiles/arch/amd64/make.defaults 18 Jan 2014 01:03:24 -0000 1.21
> +++ profiles/arch/amd64/make.defaults 27 Mar 2014 06:13:22 -0000
> @@ -21,17 +21,17 @@ ABI="amd64"
>  # 64bit specific settings.
>  CFLAGS_amd64="-m64"
>  LDFLAGS_amd64="-m elf_x86_64"
> -CHOST_amd64="x86_64-pc-linux-gnu"
> +CHOST_amd64="${CHOST}"
>  
>  # 32bit specific settings.
>  CFLAGS_x86="-m32"
>  LDFLAGS_x86="-m elf_i386"
> -CHOST_x86="i686-pc-linux-gnu"
> +CHOST_x86="i686-gentoo%multilib-linux-gnu"
Using percent sign here looks like asking for trouble at some point.
I don't see why you can't use plain 'gentoomultilib' that is more
fool-proof.

>  # 64-32bit specific settings.
>  CFLAGS_x32="-mx32"
>  LDFLAGS_x32="-m elf32_x86_64"
> -CHOST_x32="x86_64-pc-linux-gnux32"
> +CHOST_x32="x86_64-gentoo%multilib-linux-gnux32"
>  
>  # 2006/10/24 - Simon Stelling <[hidden email]>
>  # They are masked, but we can enable them anyway for those who have



--
Best regards,
Michał Górny

signature.asc (985 bytes) Download Attachment
12345