Hi Simon,

On Thu, Dec 08, 2022 at 05:20:20PM +0100, Simon Richter wrote:

I've also started work on getting usrmerge back into a sensible state, current progress is at

https://salsa.debian.org/sjr/dpkg/-/tree/wip-canonical-paths

Thank you. I've looked into this approach and I don't think I fully
understand the intended semantics yet. Can you verify whether my
understanding is correct and extend it where necessary?

Binary packages may ship a new file in control.tar called "canonical".
This file must contain an even number of lines. Each line must be an
absolute path. Odd lines identify locations of symbolic links. Even
lines identify the corresponding link target.

It is unclear to me whether more than one package may ship such a
canonical file. In case only one package may ship a canonical file, it
is unclear to me whether a canonical file can be moved between packages.

When dpkg encouners a canonial file, it will not rewrite its on-disk
database structures, but load the canonical mapping into memory. dpkg
will not move files around nor turn relevant directories into symbolic
links.

I'd like to avoid a separate tool like dpkg-divert, because we need to be able to set up aliases without being able to run maintainer scripts, so
these are useful from (c)debootstrap when setting up a foreign architecture.

While this is a nice property, I do not think that it is required. The
way this is implemented in debootstrap is adding extra scripts for
setting up the symbolic links. And for mmdebstrap, you can choose
between a hook achieving something similar and installing the usrmerge
package, which performs the conversion at postinst time. I agree that
this state of affairs is suboptimal, but at the same time it doesn't
seem like the sky is falling. Why do you think that it must be possible
to set up aliases without running maintainer scripts? It seems like I am missing some important aspect.

Ideally we'd be able to get rid of any special usrmerge handling in other tools that way as well, and gain some infrastructure that will also be
useful for declarative diversions.

Do I understand correctly that you'd like (some component of) dpkg to
perform the moving of files and creation of symlinks?

The obvious interface breakage would be dpkg-query, which would probably behave like

$ dpkg-query -S /usr/bin/ls
coreutils: /bin/ls

but we have all the relevant file names handy at this point so we can have a different format as well.

I think that dpkg -S is neither critical nor is there a way for it to
not break. If you ask it for some file, it can either return the
canonical location of a file or the actual location. Depending the use
one or the other may be desired, so some uses do get broken in any case.
You just choose its behaviour here. I think this is totally fine.

Transition would be a dpkg version that conflicts with usrmerge, and a systemd package that declares the alias and pre-depends on a recent enough version of dpkg, that will work for both upgrades and new installations.

Why do you think that systemd needs to be entangled into this? I note
that systemd is not essential. Therefore having systemd set up aliasing
would not implement the CTTE decision, becaue an essential chroot would
then be unmerged.

I think this is fixable by not touching systemd. Instead, usrmerge can
acquire the canonical table. If this method also does the conversion,
the conversion code could disappear from the usrmerge package and it
could be merged with usr-is-merged. Do you consider this adaption
reasonable? Another possible candidate for the canonical table could be base-files.

If dpkg external to the new installation is used, the files will be moved during unpacking when the systemd package is encountered, if the files are first manually unpacked, and then overwritten later during a second round, that should still work (but we need to be aware of this case).

I think you are implicitly referrig to mmdebstrap here. Your proposal implements an important (to me) property: A patched dpkg will continue
to be able to install packages on unmerged chroots.

The most annoying part will be identifying all the corner cases, like crossgrades from ppc32 to ppc64, or from Debian to Devuan and back, and writing test cases for those.

As much as I would like to solve each and every problem, corner cases
are a problem already. We noticed some of them when usrmerge became
mandatory and things started breaking again. At the same time, the
encountered corner cases were handled quite quickly. The same approach
seems viable here to me.

From my point of view, the really important property is that upgrades

that happen to involve file moves between locations and packages no
longer accidentally overwrite or delete files. As far as I understand
your proposal, such a situation is prevented by raising an error for
conclicts between canonical and non-canonical locations. It can be
avoided by adding relevant Breaks and Replaces then.

Let me also compare your patch to my vapor ware. There is quite some similarity, so I'll focus on the differences instead.

Rather than add a new option to dpkg for setting up aliases, you turn
them into a canoical mapping shipped in a binary package. As a
consequence, canonical mappings are a feature that administrators cannot
use with ease, but it can be used by packages without invoking
maintainer scripts unlike my proposal.

An aspect that I left open as a question was whether the addition of
aliases was something reversible. In uau's patch, this is irreversible
as it modifies the dpkg database on the fly. In yours, it becomes
reversible.

Do you consider this is accurate?

Can you also share your long term vision of this? Consider the time when
this would be merged and we'd update all packages to no longer ship
files in aliased directories. Then we wait a release. Should that
canonical file go away at that point and just leave the symbolic links
around or do we keep the canonical file forever?

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)