Package: kmod-udeb,busybox-udeb
Severity: serious
Justification: file conflict
X-Debbugs-Cc: Cyril Brulebois <kibi@debian.org>, debian-boot@lists.debian.org

Hi Cyril,

On Sat, Jan 06, 2024 at 04:31:44AM +0100, Cyril Brulebois wrote:

d-i daily builds now FTBFS everywhere due to the merge-usr step, with
the following error:

merge-usr "./tmp/cdrom/tree"
error: merge target 'usr//sbin/depmod' is a symlink

It would be nice if you could investigate and suggest what fix(es) would
be best here. I suppose it could be some fallout from the kmod upload
which included some changes of yours, hence the ping.

https://tracker.debian.org/news/1491892/accepted-kmod-31-1-source-into-unstable/

Thank you for the report. In investigating it, I note that reproducing
the failure was unexpectedly difficult. debian-installer fails to build
from source in unstable for unrelated reasons. I request that you update
the package more frequently and keep it in a buildable state to ease QA
efforts of others.

It turns out that /sbin/depmod points to /bin/busybox and
/usr/sbin/depmod points to ../bin/kmod. As such, merge-usr does not have
a good idea of what to do and rightly refuses to continue. These two
files are to be considered a file conflict according to the Debian
policy as they only differ in aliasing. I argue that this was a bug
before merged-/usr and merged-/usr just makes it visible now. The
underlying conflict has existed earlier and was resolved arbitrarily
dependent on the order of unpacks.

The real solution here is to agree on whether busybox-udeb or kmod-udeb
is supposed to provide depmod and only ship it once. Since there is no
other consumer of udebs, it does not make sense to install it in both.

Cyril, please reassign the bug to the right package (i.e. the one that
should not be providing depmod).

I also recommend to establish QA for all udebs to automatically detect,
report and address such conflicts as they evidently cause undefined
behaviour otherwise. That can be as simple as collecting file lists of
all udebs and comparing them.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jan 06, 2024 at 09:01:12AM +0100, Helmut Grohne wrote:

I also recommend to establish QA for all udebs to automatically detect, report and address such conflicts as they evidently cause undefined
behaviour otherwise. That can be as simple as collecting file lists of
all udebs and comparing them.

This seems like a more generic problem. I downloaded all amd64 udebs and
the following files (normalized to account for aliasing) pose a
conflict:

./bin/netcfg
./bin/ptom
./bin/wget
./etc/console-setup/remap.inc
./lib/base-installer.d/40netcfg
./lib/finish-install.d/55netcfg-copy-config
./sbin/depmod
./sbin/insmod
./sbin/lsmod
./sbin/modinfo
./sbin/modprobe
./sbin/rmmod
./usr/share/console-setup/charmap_functions.sh ./usr/share/fonts/truetype/noto/NotoSansSinhala-Bold.ttf ./usr/share/fonts/truetype/noto/NotoSansSinhala-Regular.ttf ./usr/share/fonts/truetype/noto/NotoSerifGujarati-Bold.ttf ./usr/share/fonts/truetype/noto/NotoSerifGujarati-Regular.ttf ./usr/share/keymaps/i386/qwerty/it.kmap.gz ./usr/share/keymaps/i386/qwerty/jp106.kmap.gz ./usr/share/keymaps/i386/qwertz/mac-usb-de-latin1-nodeadkeys.kmap.gz ./usr/share/keymaps/i386/qwertz/mac-usb-de-latin1.kmap.gz ./usr/share/keymaps/i386/qwertz/mac-usb-de_CH.kmap.gz ./usr/share/keymaps/i386/qwertz/mac-usb-fr_CH-latin1.kmap.gz ./usr/share/keymaps/i386/qwertz/mac-usb-pt-latin1.kmap.gz ./usr/share/keymaps/include/compose.latin1.inc.gz

So for busybox and kmod, it's not just depmod, but all the others as
well. It's also busybox-udeb vs wget-udeb and a bunch of others though.
All of the aliased ones will explode sooner or later.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

06.01.2024 11:40, Helmut Grohne:

On Sat, Jan 06, 2024 at 09:01:12AM +0100, Helmut Grohne wrote:

I also recommend to establish QA for all udebs to automatically detect,
report and address such conflicts as they evidently cause undefined
behaviour otherwise. That can be as simple as collecting file lists of
all udebs and comparing them.

This seems like a more generic problem. I downloaded all amd64 udebs and
the following files (normalized to account for aliasing) pose a
conflict:

From this list, only a few utilities are from busybox, namely wget and module utilities (depmod/insmod/lsmod/modinfo/modprobe/rmmod).

My initial plan - with regular busybox package and with busybox udeb - is to provide most things in busybox, so that other packages don't need to ship
udeb packages and the whole thing (be it d-i or initrd) is small.

Yes, some utils in busybox aren't as good as regular implementations. For example, I just found out busybox's xz does not perform compression, only decompression (-d option is mandatory). Or #1003757 - missing functionality
in busybox ip. Still, overall, it is enough for most things. BTW, it looks like with compressed kernel modules, busybox m-i-t needs some (albiet minor) tweaks (it works but kernel produces warnings when busybox tries to load a module).

Unfortunately this didn't work out for one reason or another. One of the reasons is perhaps #921556, where original util does more than needed but busybox didn't implement the unnecessary functionality.

This needs to be thought about at a more general level. Including initrd
stuff (if we still need it, instead of relying on mkosi-initrd). I use
my own initrd for a good reason, and this one does not include 2 or even
3 libc as debian does..

/mjt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

09.04.2024 16:48, Cyril Brulebois wrote:

Marco d'Itri <md@Linux.IT> (2024-04-09):

Yes. Nowadays kmod has many more features related to compressed modules
and verification of signatures.
Can we agree that kmod should provide these programs for d-i?
Or can the d-i maintainers just tell us what they want?

I meant to come back to this after experimenting, then things happened…

I picked kmod at the time because it worked, and because busybox didn't
work, which I summed up in:
https://salsa.debian.org/installer-team/debian-installer/-/commit/450daf0bd24ee94d4f466ab65908c079ef795145

(plus follow-up commit, woopsie
https://salsa.debian.org/installer-team/debian-installer/-/commit/69777be465c5d0210d16159a456ab88535513a07
)

I'm fine with sticking to kmod regarding module support in d-i. I'm not
sure we should keep support in two different modules, so dropping it
from busybox would work for me. Others might have different views on
this, though.

So, should I disable module utils in busybox-udeb now?
Wanted to spare some time on busybox, this bug report come in.

Is kmod udeb ready and used in d-i already, or does it need some
prep first?

Thanks,

/mjt

--
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E 9D8B E14E 3F2A 9DD7 9199 28F1 61AD 3D98 ECDF 2C8E
Old key: rsa2048/457CE0A0804465C5 6EE1 95D1 886E 8FFB 810D 4324 457C E0A0 8044 65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Marco d'Itri <md@linux.it> (2024-04-26):

On Apr 26, Michael Tokarev <mjt@tls.msk.ru> wrote:

So, should I disable module utils in busybox-udeb now?

I think so.

I haven't gotten any requests / seen any reasons to keep it; so, yes,
please feel free to remove it whenever is convenient for you.

Is kmod udeb ready and used in d-i already, or does it need some
prep first?

AFAIK it works.

Absolutely, that's been working since the small xz-utils tweak (the udeb addition, not the backdoor thing).


Cheers,
--
Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmYrooQACgkQ/5FK8MKz VSBlng//Xf4nzElw2VxzKtfc4elxwQ4DobilYnCnei4xMDyH+qTB+SmtApQM/VIK jn92jJZxGTeEw/jjpLrT6fbdGT8uitq9wxPpDRISPfLwHWyVzRknwqKIZbmcP0JY 3nWg7LBqY+LA+q1nYtiPIvwG6VcbiulC0uaweDbPx8i7qv7LOudrw/52PgSNBJyH nGs9nBjy63X9RIXJ+Lr8uDPKNg3HaEoMArw2eWKJx9E1N+sGJl57wq3k4ppGTrqU iFpx09evo4PjfgCW6rhOTJV+x5sI/0q6uAD/u3bxN83+1Vh0CbfGM9NkpuGpHlMQ KQnxOIUVl1Ay8Fbs/UXU1HnWmsRzHDyDw2E9RkElXBbJKTERSx4p7x8dKQr4edaa qMcdpeQrzICPND3C1jVU6PBx0vEdmrY1MgTM+TXYBIsIXYSwNOGV5JmtBxEli4Q6 Eg53xoT4wj/OoI/BafJFkYuejC5g8OThocB2p+aDuX4M2VHGIJXBLkXGlkOGXYHE B1zvsNgdB9EMCoFikLkc4zQg+4y6w94ya0cR7/3Z1FhqXDifp/mCGWeQBMznYOUy ryMwycafHaf67kIqsWBB5v7MAgVbDr9hRACVCzUyvVtM7xtEPGaTqdvX4y0OJ/pl g8ywlSxAI0X2HkmCKQVq6vIh1BBM+6k5JcvR4j1et8jjOVoJbAU=
=s802
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
*