Hi Bastian,

On Mon, Mar 04, 2024 at 12:30:09PM +0100, Bastian Blank wrote:

On Mon, Mar 04, 2024 at 12:07:15PM +0100, Matthias Klose wrote:

On 04.03.24 11:29, Bastian Blank wrote:

On Mon, Mar 04, 2024 at 08:53:11AM +0100, Matthias Klose wrote:

However the links in /usr/DEB_HOST_GNU_TYPE/include are missing.

Please be a bit more precise, there are no symlinks in this directory.
| # dpkg -S /usr/alpha-linux-gnu/include/asm/a.out.h
| linux-libc-dev-alpha-cross: /usr/alpha-linux-gnu/include/asm/a.out.h
| # find /usr/alpha-linux-gnu/include/ -type l
| #

yes, that is the problem. the cross gcc expects these headers in /usr/alpha-linux-gnu/include, not in the header location for the host.

Please show your problem with a log, my crystal ball is broken.

This very much was not obvious to me either. I've now talked to Matthias
and believe I can explain the problem.

The packaged gcc cross toolchain uses a sysroot during its own build
still. As it is implemented now, it searches /usr/<triplet>/include, but
not /usr/include/<multiarch>. So quite fundamentally, the Provides that
we two agreed do break the build of cross toolchains right now.

Arguably, a cross toolchain build should probably search /usr/include/<multiarch>. I went back and forth a bit with Matthias
about whether we could add this and did not fully understand his
reasons, but there is one technical reason we want to avoid it for now.
We can have both libc6-dev:TARGET and libc6-dev-TARGET-cross installed
and these packages can have differing versions. When that happens and we
search both /usr/<triplet>/include and /usr/include/<multiarch>, we'd
mix two glibc versions with usually bad results (been there).

While we might consider adding /usr/include/<multiarch> to the cross
toolchain build search path later, it is not something we can do now and
before doing that, we need to better understand Matthias' reasons for
keeping these separate as well.

The other aspect here is that it is not sufficient to add /usr/include/<multiarch> to the search path as you also need
/usr/include to get a complete linux kernel headers experience. We
definitely do not want to add /usr/include, because that is known to
misguide configure tests performed for the target architecture. In
principle, we could extend the symlink farm in linux-libc-dev to also
have a lot of /usr/include/<multiarch>/linux -> ../linux symlinks
(assuming that no other package ever installs to /usr/include/linux,
which is a property violated by aufs-dev and oss4-dev).

So at least for now, I am convinced that we will need
/usr/<triplet>/include to be provided by the package providing linux-libc-dev-arch-cross.

As these are only necessary for building the cross toolchains, we
probably don't want these in the main linux-libc-dev package. So how
about adding a linux-libc-dev-cross package with yet more symlinks? Then
we can move the provides over to the linux-libc-dev-cross package and
that way repair the cross toolchain builds.

I requested that linux-libc-dev provides these for bootstrapping to know
which architectures linux-libc-dev actually supports. I don't need these provides exactly, I just need a mechanism to answer the question "Does linux-libc-dev work for a particular architecture?" from the binary
package metadata, so we might just change the Provides there to linux-libc-dev-arch dropping the -cross suffix that traditionally
identified packages putting stuff into /usr/<triplet>. Does that sound reasonable to you?

That still leaves the question of which package would have to build that
new linux-libc-dev-cross. The two obvious answers are linux and cross-toolchain-base. Do you have a preference here?

I hope this all makes more sense now.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Bastian,

On Mon, Mar 04, 2024 at 11:04:22PM +0100, Bastian Blank wrote:

Arguably, a cross toolchain build should probably search /usr/include/<multiarch>. I went back and forth a bit with Matthias
about whether we could add this and did not fully understand his
reasons, but there is one technical reason we want to avoid it for now.
We can have both libc6-dev:TARGET and libc6-dev-TARGET-cross installed
and these packages can have differing versions. When that happens and we search both /usr/<triplet>/include and /usr/include/<multiarch>, we'd
mix two glibc versions with usually bad results (been there).

But this is a search path. If a file exists in one, the second one is
not found. So nothing can happen even from version skew.

The problem arises in the reverse sense. If a file does not exist in
one, it is searched in the second and erroneously may be found. That may
make tests pass that should not pass and typically causes a link failure
later. While I do not have a concrete example at hand, I have seen this
pattern repeatedly and generally favour moving stuff out of /usr/include
to avoid this kind of confusion that causes difficult to debug problems.
This also motivates #798955 (in addition to the problem with file
conflicts).

The other aspect here is that it is not sufficient to add /usr/include/<multiarch> to the search path as you also need
/usr/include to get a complete linux kernel headers experience. We definitely do not want to add /usr/include, because that is known to misguide configure tests performed for the target architecture.

We are talking about the toolchain itself. What configure tests could
that be? Or is that premature optimization of the gcc build?

The one case I really do remember quite well is sanitizers. These should
not be enabled in the earlier toolchain stages and failing to disable
them tends to cause linker failures. I don't think it matches the
concrete situation exactly though. You also make a good case in your
followup reporting that gcc-13-cross actually builds.

You just said that the search path used during the build of the
toolchain and the one for everything else are unrelated. So you are
free to create $BUILD/tmp-include with symlinks for asm, asm-generic,
linux.

The toolchain as installed already finds all headers. So I still don't
see why we need this in the final system.

I find this argument fairly convincing and hope Matthias also does.

Thank you

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Helmut

On Tue, Mar 05, 2024 at 09:50:27AM +0100, Helmut Grohne wrote:

The problem arises in the reverse sense. If a file does not exist in
one, it is searched in the second and erroneously may be found. That may
make tests pass that should not pass and typically causes a link failure later.

But you want /usr/include to be found. Otherwise you would not be able
to use most of the libraries for cross compiling.

. While I do not have a concrete example at hand, I have seen this pattern repeatedly and generally favour moving stuff out of /usr/include
to avoid this kind of confusion that causes difficult to debug problems.
This also motivates #798955 (in addition to the problem with file
conflicts).

In this case here, this would just find kernel headers for a different
version. Those are essentially a headers only library, nothing is
available for linking. And all the headers provided in /usr/include are
the same for all architectures.

So moving stuff out of /usr/include might be a good idea if the -dev
package is itself arch dependent.

The one case I really do remember quite well is sanitizers. These should
not be enabled in the earlier toolchain stages and failing to disable
them tends to cause linker failures. I don't think it matches the
concrete situation exactly though. You also make a good case in your
followup reporting that gcc-13-cross actually builds.

Yep. My problem is: I tested stuff, not everything of course, and did
not see any breakage. Also I checked the values I know could influence
that and they say it is fine. So at some point I have to assume it is
not broken, as exhaustive search is simply not possible.

The only statement in this bug report is now: it is located in a
different location, so it is broken. No single piece of evidence is
shown, like broken builds or some other ways to see the brokeness.

Regards,
Bastian

--
A princess should not be afraid -- not with a brave knight to protect her.
-- McCoy, "Shore Leave", stardate 3025.3

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)