I spoke to Russ and we're both of the view that we should document
multiarch piecemeal. Let's begin by getting a definition of the
Multi-Arch: field into ch. 5 of policy.
I have pushed a new branch to the Debian policy repo named bug749826-spwhitton. On that branch I've committed a slightly reworked
form of your draft text.[1] Please review the diff. Here are some comments/issues:
- I substantially shortened your text. Let me know if you think I went
too far.
- Previously I was worried about defining 'interface' but I've found
another place where policy uses this word without defining it, and I
don't think it needs to be changed in either place.
- I couldn't figure out how to include this text, because I didn't
understand it:
For instance, using dpkg --print-architecture can be used to emit the
native architecture even though dpkg is marked Multi-Arch:
foreign. Similarly, calling pkg-config (without a prefix) will behave
differently on different architectures as its search path is
architecture-dependent even thoug pkg-config is marked Multi-Arch:
foreign.
Are you saying that packages that depend or implicitly depend on dpkg
or pkg-config cannot be Multi-arch: foreign, although dpkg and
pkg-config themselves are Multi-arch: foreign? Why are dpkg and
pkg-config Multi-arch: foreign, if they provide these
architecture-dependent interfaces?
- I didn't include your TODO about README.multiarch; let me know whether
you have a more concrete idea about the purpose of that file
- after we've got text documenting the other possible values of the
Multi-Arch: field, we might want to promote the list of things to
consider out of the Multi-Arch: foreign subsubsection. It should
become clear once we've got that other text together.
Thank you again for your work so far.
On Sun, Aug 20 2017, Helmut Grohne wrote:
A common theme with such cases is to resort to `Multi-Arch: allowed`
(e.g. make), but that has the downside of requiring most consumers to attach the :any annotation and that it can never be switched back
(because :any dependencies on packages not marked M-A:allowed are unsatisfiable).
* If there is such a conflict, but the relevant packages are not
coinstallable due to package relations, is that ok? Example: libc6
has such a conflict on /lib/ld.so.1 for mips and mipsel.
(Presently, you get an unpack error here.)
If libc6's use is legitimate then it seems we'd need to include this as
an exception.
I think "the files installed by ``Architecture: all`` packages always provide architecture-independent interfaces." is too broad. The counter example is haskell-devscripts-minimal. This needs to be weakened
somehow.
For instance, the policy should make it
clear that marking libmdds-dev `Multi-Arch: foreign` (fictional, see #843023) would be a policy violation.
- I couldn't figure out how to include this text, because I didn't
understand it:
For instance, using dpkg --print-architecture can be used to emit the
native architecture even though dpkg is marked Multi-Arch:
foreign. Similarly, calling pkg-config (without a prefix) will behave
differently on different architectures as its search path is
architecture-dependent even thoug pkg-config is marked Multi-Arch:
foreign.
Are you saying that packages that depend or implicitly depend on dpkg
or pkg-config cannot be Multi-arch: foreign, although dpkg and
pkg-config themselves are Multi-arch: foreign? Why are dpkg and
pkg-config Multi-arch: foreign, if they provide these
architecture-dependent interfaces?
Those are very good questions and clarifying them will lead to a better understanding of what we have to put into policy. You do understand that "dpkg --print-architecture" is part of dpkg's interface. Yet its out
varies with its architecture. Taking this strictly would indeed imply
that dpkg is wrongly marked. Similarly, running pkg-config may result in architecture-dependent paths and thus our strict interpretation would
result in rejecting the foreign marking.
A common theme with such cases is to resort to `Multi-Arch: allowed`
(e.g. make), but that has the downside of requiring most consumers to
attach the :any annotation and that it can never be switched back
(because :any dependencies on packages not marked M-A:allowed are unsatisfiable).
This is where I thought about README.multiarch:
- I didn't include your TODO about README.multiarch; let me know whether
you have a more concrete idea about the purpose of that file
It can document assumptions one makes about users of a package. For
instance, we expect dpkg users to use `dpkg --print-architecture` diagnostically only. Similarly, we expect that package builds call
pkg-config if they mean the build architecture and they need to call $(DEB_HOST_GNU_TYPE)-pkg-config if they mean the host architecture.
Indeed that happens automatically for autotools projects that happen to
use PKG_CHECK_MODULES or PKG_PROG_PKG_CONFIG (i.e. most). It also
happens for cmake when built with dh_auto_build.
Let me give a counter example to illustrate more of the point. haskell-devscripts-minimal is an `Architecture: all` package with some
shell scripts. Sounds like a good candidate for `Multi-Arch: foreign`.
When you look at /usr/share/haskell-devscripts/Dh_Haskell.sh though, you
see that functions such as cpu(), os(), etc. specifically introspect the build architecture by using the build architecture ghc. Such usage is
not ok for `Multi-Arch: foreign` (#769377).
I believe that policy should encourage some uniform way to document the intended interface as we have several cases where this is not obvious. README.multiarch may be that way. In particular, using a package in a
way not permitted by such README.multiarch would need to be a policy violation on its own. For instance, one could depend on a shared library
and declare it an implementation detail. Relying on the transitive
dependency would then be considered a policy violation.
- after we've got text documenting the other possible values of the
Multi-Arch: field, we might want to promote the list of things to
consider out of the Multi-Arch: foreign subsubsection. It should
become clear once we've got that other text together.
Indeed, documenting `Multi-Arch: same` may be easier (or not). For the purpose of defining it, we shall call Debian binary packages for
different architectures with equal binary package name and version "instances" of a package. I currently see the following requirements:
* It must not be used on `Architecture: all` packages (though I wish
you could ;).
* Given any two instances of a package and any filename, that filename
must be non-existent in at least one package or the type (directory /
regular file / etc.) must match and when the filename refers to a
regular file, the contents must be bitwise equal in both instances.
This point deserves some more thought as it has some pitfalls:
* If there is such a conflict, but the relevant packages are not
coinstallable due to package relations, is that ok? Example: libc6
has such a conflict on /lib/ld.so.1 for mips and mipsel.
(Presently, you get an unpack error here.)
* If you rebuild the source package with a very different
installation set (i.e. much newer Build-Depends), does it still
have to match with older instances? Example: #825146. What
divergence in installation sets is ok?
(A simple way to satisfy this requirement is to use
architecture-dependent paths exclusively. That works except for
/usr/share/doc/$pkg.)
* The maintainer scripts must handle multiple configuration and
multiple deconfiguration correctly. In particular, a package can be
purged for one architecture while being installed for another.
Example: #682420.
(A simple way to satisfy this requirement is to not ship maintainer
scripts.)
* Source packages carrying any binary package marked `Multi-Arch: same`
must always be binNMUed in lock-step. (Presently violated e.g. by
libselinux1)
Ah right and you asked for a review of the branch. ;)
I think "the files installed by ``Architecture: all`` packages always
provide architecture-independent interfaces." is too broad. The counter example is haskell-devscripts-minimal. This needs to be weakened
somehow.
Your shortening of my draft has significantly cut precision and
improved readability. It is difficult to strike a useful balance here
and I am certainly biased towards precision. I'm unsure how to proceed
here. I believe that your current form will cause discussion and
dispute among policy readers. For instance, the policy should make it
clear that marking libmdds-dev `Multi-Arch: foreign` (fictional, see
#843023) would be a policy violation.
So the current text falls short on objections e.g. Bill Allombert raised
on the reproducible issue: Can I mechanically check policy requirements?
It is actually worse here: Given a Multi-Arch bug report, it is
difficult to determine whether your draft supports it due to the lack of precision.
That seems like it might be a bug (or design flaw if you prefer). If a package (build-)depends on foo:any, it is saying "I am only using the arch-indep parts of foo's interface", whatever those are.
Perhaps a dependency on foo:any by (for example) bar:mips should
always be satisfiable by foo:mips (as though the :any had been omitted), regardless of foo's multi-arch status? This would bring it back to the
same meaning as omitting the :any, in the trivial case where only one architecture is enabled.
Perhaps a dependency on foo:any should be satisfiable by any instance
of foo that is Multi-Arch: foreign? (In this case the :any is completely redundant, because foreign sets up a similar situation from the other end)
I think "the files installed by ``Architecture: all`` packages always provide architecture-independent interfaces." is too broad. The counter example is haskell-devscripts-minimal. This needs to be weakened
somehow.
I would argue that these interfaces are architecture-independent from
the perspective of the package's (lack of) architecture. What they
are not independent of is the *build machine* architecture, just like
running uname -m or inspecting /proc/cpuinfo aren't independent of the
build machine architecture. This is certainly a problem for cross-compilation, but it isn't the same issue as in dpkg or pkg-config, where the architecture for which dpkg or pkg-config was built gets
hard-coded into its installed files (as the output of --print-architecture
or part of the default search path, respectively).
For instance, the policy should make it
clear that marking libmdds-dev `Multi-Arch: foreign` (fictional, see #843023) would be a policy violation.
It is not clear to me that doing so *should* be a policy violation. If libmdds-dev contains only headers (no shared or static library), and it exposes architecture-independent libboost-dev headers (but no Boost
shared or static library), is there really anything wrong with having libboost-dev from "the wrong architecture"?
Rather than introduce the new terminology 'intended interface', which we would definitely have to define, how about something like this:
If all a package's architecture-dependent interfaces are listed in
README.multiarch, the package is not considered to have any
architecture-dependent interfaces for the purposes of determining
whether it may be labelled Multi-Arch: foreign.
If libc6's use is legitimate then it seems we'd need to include this as
an exception.
* If you rebuild the source package with a very different
installation set (i.e. much newer Build-Depends), does it still
have to match with older instances? Example: #825146. What
divergence in installation sets is ok?
We could just say that it must match the instances in the target suite.
(A simple way to satisfy this requirement is to use
architecture-dependent paths exclusively. That works except for
/usr/share/doc/$pkg.)
* The maintainer scripts must handle multiple configuration and
multiple deconfiguration correctly. In particular, a package can be
purged for one architecture while being installed for another.
Example: #682420.
(A simple way to satisfy this requirement is to not ship maintainer
scripts.)
* Source packages carrying any binary package marked `Multi-Arch: same`
must always be binNMUed in lock-step. (Presently violated e.g. by
libselinux1)
Could you turn this into some commits against my branch, please?
It sounds like we need to just drop the whole bullet point.
Architecture: all packages need to be checked carefully, just like Architecture: any packages.
To my mind, the most important ways to achieve readability in this case
are
- avoid repetition
- avoid "probably", "likely" sentences.
On Sat, Sep 02, 2017 at 08:44:14AM -0700, Sean Whitton wrote:
Rather than introduce the new terminology 'intended interface', which
we would definitely have to define, how about something like this:
If all a package's architecture-dependent interfaces are listed
in README.multiarch, the package is not considered to have any
architecture-dependent interfaces for the purposes of determining
whether it may be labelled Multi-Arch: foreign.
This is not how it works. It's not like you can just mark any package Multi-Arch: foreign after saying that it is
architecture-dependent. That documentation must come with a contract
saying that reverse dependencies must not use those
architecture-dependent interfaces.
If libc6's use is legitimate then it seems we'd need to include this
as an exception.
Well, it's not exactly legitimate. It's more like unavoidable as Simon pointed out in his reply. Technically, libc6's behaviour is wrong and
causes unpack errors. The reasonable solution would be prohibiting coinstallation of libc6:mips and libc6:mipsel, but package metadata
does not allow us to do that currently (#747261 -> self-conflicts are
always ignored). The other option of removing Multi-Arch: same from
libc6 would essentially render Multi-Arch useless. So all we can do
now is pretend the issue wasn't there.
* If you rebuild the source package with a very different
installation set (i.e. much newer Build-Depends), does it still
have to match with older instances? Example: #825146. What
divergence in installation sets is ok?
We could just say that it must match the instances in the target
suite.
We could. That would render libgiac0 rc buggy for instance, because it
was built on mips64el three weeks later than on other architectures
and thus uses an incompatible gettext.
That definition is pretty annoying for bootstraps though as
replicating ancient toolchain is kinda the opposite of what
bootstrappers do.
Could you turn this into some commits against my branch, please?
I tried and ran into a new problem: I am now convinced that we cannot
just describe one Multi-Arch value after another as they do share some
common values. That "interface" aspect and architecture-constraints on dependencies is a common theme and likely deserves an introductory
text.
Yet, I am attaching what I have.
As Simon's mail demonstrates, we likely need more answers/consensus
before continuing. I'll reply in a separate mail.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 296 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 40:38:57 |
| Calls: | 6,648 |
| Files: | 12,193 |
| Messages: | 5,329,420 |