Hello,

I just wanted to report that the dpkg 1.21.11 and 1.21.10 test suite
fails on Alpine Linux Edge. The test suite passed successfully for
previous version of dpkg.

The specific test that fails is the ./t/mk.t.

The output looks as follows:

./t/mk.t ....................... Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_TYPE"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_ARCH_OS"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_ARCH_OS"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_ARCH_OS"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.
Use of uninitialized value $v{"DEB_TARGET_GNU_CPU"} in concatenation (.) or string at /home/soeren/src/aports/main/dpkg/src/dpkg-1.21.11/scripts/dpkg-architecture line 400.

These messages are repeated in an infinite loop for both dpkg 1.21.11
and 1.21.10, the test does not terminate. I suspect that this is related
to commit c15487a58cfd955ae401c872c95aa375aacd1a88 (author in CC) [1].
If I revert this commit then the mk.t test passes on Alpine Linux. Unfortunately, I don't speak perl so I can't provide a proper fix.

Please let me know if you need more information.

Greetings,
Sören Tempel

[1]: https://git.dpkg.org/cgit/dpkg/dpkg.git/commit/scripts/dpkg-architecture.pl?id=c15487a58cfd955ae401c872c95aa375aacd1a88

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi!

On Mon, 2022-12-05 at 16:03:44 +0100, Sören Tempel wrote:

I just wanted to report that the dpkg 1.21.11 and 1.21.10 test suite
fails on Alpine Linux Edge. The test suite passed successfully for
previous version of dpkg.

Thanks for the report!

The specific test that fails is the ./t/mk.t.

The output looks as follows:

[…]

These messages are repeated in an infinite loop for both dpkg 1.21.11
and 1.21.10, the test does not terminate. I suspect that this is related
to commit c15487a58cfd955ae401c872c95aa375aacd1a88 (author in CC) [1].
If I revert this commit then the mk.t test passes on Alpine Linux. Unfortunately, I don't speak perl so I can't provide a proper fix.

Please let me know if you need more information.

That's weird. I cannot reproduce this. It works in Debian (but that
could imply some hidden dependency), and I just spawned an alpine:latest
Docker image, and it passes there too with what's currently on git HEAD:

,---
/src/dpkg # cat /etc/alpine-release
3.17.0
/src/dpkg # apk --version
apk-tools 2.12.10, compiled for x86_64.
/src/dpkg # make -C scripts/ check
[…]
./t/merge_changelogs.t ......... ok
./t/mk.t ....................... ok
All tests successful.
Files=47, Tests=24295, 11 wallclock secs ( 1.11 usr 0.12 sys + 8.06 cusr 1.34 csys = 10.63 CPU)
Result: PASS
make[2]: Leaving directory '/src/dpkg/scripts'
make[1]: Leaving directory '/src/dpkg/scripts'
make: Leaving directory '/src/dpkg/scripts'
`---

If you can point a way for me to reproduce, I'm happy to dig further.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

Thanks for your quick response.

Guillem Jover <guillem@debian.org> wrote:

That's weird. I cannot reproduce this. It works in Debian (but that
could imply some hidden dependency), and I just spawned an alpine:latest Docker image, and it passes there too with what's currently on git HEAD:

,---
/src/dpkg # cat /etc/alpine-release
3.17.0
/src/dpkg # apk --version
apk-tools 2.12.10, compiled for x86_64.
/src/dpkg # make -C scripts/ check
[…]
./t/merge_changelogs.t ......... ok
./t/mk.t ....................... ok
All tests successful.
Files=47, Tests=24295, 11 wallclock secs ( 1.11 usr 0.12 sys + 8.06 cusr 1.34 csys = 10.63 CPU)
Result: PASS
make[2]: Leaving directory '/src/dpkg/scripts'
make[1]: Leaving directory '/src/dpkg/scripts'
make: Leaving directory '/src/dpkg/scripts'
`---

alpine:latest is the latest stable release. I encountered this on
alpine:edge, which is our rolling release development version (basically
Debian Sid). It is interesting that it doesn't seem to fail in
alpine:latest though. However, I can reproduce it in a fresh alpine:edge
Docker image using the following commands:

$ docker run -it alpine:edge
/ # cat /etc/alpine-release
3.17_alpha20221110
/ # apk add build-base ncurses-dev xz coreutils gzip bzip2-dev linux-headers perl zlib-dev po4a tar libmd-dev
/ # wget https://deb.debian.org/debian/pool/main/d/dpkg/dpkg_1.21.11.tar.xz
/ # tar -xvf dpkg_1.21.11.tar.xz
/ # cd dpkg-1.21.11
/ # ./configure --disable-nls
/ # make -j$(nproc)
/ # make -C scripts/ check

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

Greetings
Sören

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 2022-12-05 at 20:43:45 +0100, Sören Tempel wrote:

alpine:latest is the latest stable release. I encountered this on alpine:edge, which is our rolling release development version (basically Debian Sid). It is interesting that it doesn't seem to fail in
alpine:latest though. However, I can reproduce it in a fresh alpine:edge Docker image using the following commands:

$ docker run -it alpine:edge
/ # cat /etc/alpine-release
3.17_alpha20221110
/ # apk add build-base ncurses-dev xz coreutils gzip bzip2-dev linux-headers perl zlib-dev po4a tar libmd-dev
/ # wget https://deb.debian.org/debian/pool/main/d/dpkg/dpkg_1.21.11.tar.xz
/ # tar -xvf dpkg_1.21.11.tar.xz
/ # cd dpkg-1.21.11
/ # ./configure --disable-nls
/ # make -j$(nproc)
/ # make -C scripts/ check

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

Ah, sorry, can certainly reproduce now. This smells like a problem with
GNU make 4.4, AFAIR that release contains several behavior breaking
changes. Will see what's going on and prep a fix for 1.21.12.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi!

On Mon, 2022-12-05 at 21:16:19 +0100, Guillem Jover wrote:

On Mon, 2022-12-05 at 20:43:45 +0100, Sören Tempel wrote:

alpine:latest is the latest stable release. I encountered this on alpine:edge, which is our rolling release development version (basically Debian Sid). It is interesting that it doesn't seem to fail in alpine:latest though. However, I can reproduce it in a fresh alpine:edge Docker image using the following commands:

$ docker run -it alpine:edge
/ # cat /etc/alpine-release
3.17_alpha20221110
/ # apk add build-base ncurses-dev xz coreutils gzip bzip2-dev linux-headers perl zlib-dev po4a tar libmd-dev
/ # wget https://deb.debian.org/debian/pool/main/d/dpkg/dpkg_1.21.11.tar.xz
/ # tar -xvf dpkg_1.21.11.tar.xz
/ # cd dpkg-1.21.11
/ # ./configure --disable-nls
/ # make -j$(nproc)
/ # make -C scripts/ check

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

Ah, sorry, can certainly reproduce now. This smells like a problem with
GNU make 4.4, AFAIR that release contains several behavior breaking
changes. Will see what's going on and prep a fix for 1.21.12.

Ok, looks like this might be due to the new GNU make behavior change
where $(shell) now properly gets all exported variables, which is causing recursion. Passing -f to dpkg-arechitecture in the .mk fragment makes the
test pass, but I've to make sure this is the right fix, as that will mean externally set variables are not honored, but then that would be the case anyway with the old $(shell) behavior, so I guess not a regression.

Attached a tentative patch.

Thanks,
Guillem

From 891fff7ec1e0db5fa4a951ba22ec7a13fba80767 Mon Sep 17 00:00:00 2001
From: Guillem Jover <guillem@debian.org>
Date: Mon, 5 Dec 2022 21:47:28 +0100
Subject: [PATCH] scripts/mk: Pass -f to dpkg-architecture in architecture.mk MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

With new GNU make 4.4, the $(shell) function now properly inherits
exporter variables, which is making these lead to recursion during
expansion. Given that we were not respecting any externally set
environment variables by the user, forcing to ignore them is not a
regression (FIXME, although perhaps that behavior should actually
be fixed!).

Reported-by: S=C3=B6ren Tempel <soeren@soeren-tempel.net>
---
scripts/mk/architecture.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/mk/architecture.mk b/scripts/mk/architecture.mk
index c11cada16..8480411f6 100644
--- a/scripts/mk/architecture.mk
+++ b/scripts/mk/architecture.mk
@@ -4,7 +4,7 @@
=20
dpkg_lazy_eval ?=3D $$(or $$(value DPKG_CACHE_$(1)),$$(eval DPKG_CACHE_$(1=
) :=3D $$(shell $(2)))$$(value DPKG_CACHE_$(1)))
=20
-dpkg_architecture_setvar =3D export $(1) ?=3D $(call dpkg_lazy_eval,$(1),d= pkg-architecture -q$(1))
+dpkg_architecture_setvar =3D export $(1) ?=3D $(call dpkg_lazy_eval,$(1),d= pkg-architecture -f -q$(1))
=20
$(foreach machine,BUILD HOST TARGET,\
$(foreach var,ARCH ARCH_ABI ARCH_LIBC ARCH_OS ARCH_CPU ARCH_BITS ARCH_EN= DIAN GNU_CPU GNU_SYSTEM GNU_TYPE MULTIARCH,\
--=20
2.38.1

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 2022-12-05 at 21:52:38 +0100, Guillem Jover wrote:

Ok, looks like this might be due to the new GNU make behavior change
where $(shell) now properly gets all exported variables, which is causing recursion. Passing -f to dpkg-arechitecture in the .mk fragment makes the test pass, but I've to make sure this is the right fix, as that will mean externally set variables are not honored, but then that would be the case anyway with the old $(shell) behavior, so I guess not a regression.

Attached a tentative patch.

Please disregard the patch, while the new GNU make behavior might
surface this, the fact that reverting the commit that you bisected
fixes it (too?) means there is definitely something wrong with the
new code. I'll check what's going on later today when I've got a
moment.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi!

On Mon, 2022-12-05 at 20:43:45 +0100, Sören Tempel wrote:

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

So this should be properly fixed now in dpkg 1.21.12 which I released yesterday. Thanks! :)

While I'm checking the Alpine packaging I noticed some things that
could be improved:

* The dpkg-checkbuilddeps test workaround does not seem to be needed
anymore? (I recall fixing that, and at least it seemed to pass on
the alpine:edge Docker image).
* Might need a makedepends on xz-dev (and remove the xz from depends?).
* Might need a depends on xz, gzip and bzip2 for $pkgname-dev, not
sure whether one can specify fine-grained dependencies only for
that package though? Perhaps within the dev() function?
* The dpkg-dev might need a depends on dpkg.
* There should be no need to create the updates/ dir, nor touch the
status nor available file anymore, they are supposed to be created
on demand now.
* The dpkg-dev package ships the headers, .pc file and similar but
the static library (as the library is not stable yet and there is
no shared library yet), gets removed in package(). So probably stop
removing the static library? Some packages (at least in Debian)
make use of it.
* The dpkg-buildflags and dpkg-genbuildinfo should be moved to
dpkg-dev. The same with the usr/share/dpkg/*.mk files.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 2022-12-06 at 16:45:40 +0100, Guillem Jover wrote:

On Mon, 2022-12-05 at 20:43:45 +0100, Sören Tempel wrote:

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

So this should be properly fixed now in dpkg 1.21.12 which I released yesterday. Thanks! :)

While I'm checking the Alpine packaging I noticed some things that
could be improved:

* The dpkg-checkbuilddeps test workaround does not seem to be needed
anymore? (I recall fixing that, and at least it seemed to pass on
the alpine:edge Docker image).

Ah, sorry, disregard this one. I think I might have installed dpkg on
the Docker image and that's why it was passing. But I'll fix this for
dpkg 1.21.13.

* Might need a makedepends on xz-dev (and remove the xz from depends?).
* Might need a depends on xz, gzip and bzip2 for $pkgname-dev, not
sure whether one can specify fine-grained dependencies only for
that package though? Perhaps within the dev() function?
* The dpkg-dev might need a depends on dpkg.
* There should be no need to create the updates/ dir, nor touch the
status nor available file anymore, they are supposed to be created
on demand now.
* The dpkg-dev package ships the headers, .pc file and similar but
the static library (as the library is not stable yet and there is
no shared library yet), gets removed in package(). So probably stop
removing the static library? Some packages (at least in Debian)
make use of it.
* The dpkg-buildflags and dpkg-genbuildinfo should be moved to
dpkg-dev. The same with the usr/share/dpkg/*.mk files.

I've also now (pushed to git) fixed all the compiler warnings that
appear during the build on Alpine. :)

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello!

Guillem Jover <guillem@debian.org> wrote:

On Mon, 2022-12-05 at 20:43:45 +0100, Sören Tempel wrote:

I haven't tried reproducing the failure on alpine:latest yet so please
let me know if you are also not able to reproduce the hang on alpine:edge.

So this should be properly fixed now in dpkg 1.21.12 which I released yesterday. Thanks! :)

Can confirm that this has been fixed in 1.21.12, thanks for the quick fix!

While I'm checking the Alpine packaging I noticed some things that
could be improved:

Thank you for your input, appreciate it! I implemented most of this
already. Just some minor follow up question below.

* Might need a makedepends on xz-dev (and remove the xz from depends?).

The xz dependency was added in 2020 with the commit message "dpkg needs
xz to create packages and busybox xz only support extraction" [1] is
this still the case? If so: we do need it to be a runtime dependency.

* Might need a depends on xz, gzip and bzip2 for $pkgname-dev, not
sure whether one can specify fine-grained dependencies only for
that package though? Perhaps within the dev() function?

It is possible to just define dependencies for the dev() package. Could
you explain which part of the dev subpackage needs these compression
tools though? Unfortunately, Alpine does not have an optional dependency concept so if a dependency is just needed by a particular script in a
package then we tend not to add it.

* The dpkg-dev might need a depends on dpkg.

Could you elaborate why this is necessary?

Thanks again!

Greetings,
Sören

[1]: https://git.alpinelinux.org/aports/commit/?id=785438d9b145d47144f450a0ec4567f76e9ae330

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi!

BTW, I've now also pushed fixes to git HEAD, for the test failures with dpkg-checkbuilddeps, and fixed the test suite to not require coreutils,
which will be part of 1.21.13.

On Wed, 2022-12-07 at 19:50:08 +0100, Sören Tempel wrote:

Guillem Jover <guillem@debian.org> wrote:

* Might need a makedepends on xz-dev (and remove the xz from depends?).

The xz dependency was added in 2020 with the commit message "dpkg needs
xz to create packages and busybox xz only support extraction" [1] is
this still the case? If so: we do need it to be a runtime dependency.

Sorry the "action items" for this were probably more confusing than
helpful. See below.

* Might need a depends on xz, gzip and bzip2 for $pkgname-dev, not
sure whether one can specify fine-grained dependencies only for
that package though? Perhaps within the dev() function?

It is possible to just define dependencies for the dev() package. Could
you explain which part of the dev subpackage needs these compression
tools though? Unfortunately, Alpine does not have an optional dependency concept so if a dependency is just needed by a particular script in a
package then we tend not to add it.

To handle binary .deb packages, the dpkg-deb program (via libdpkg.a) can
either use the compression libraries or the tools, dpkg uses dpkg-deb
as a backend to extract the contents when installing binary packages but
that should in general not be relevant on Alpine. I think using the shared libraries is in general better, but I'd only strongly recommend that for a system where dpkg is the native package manager, otherwise do as you deem
best. But given that the packaging is already linking against the other
shared libraries (bzip2-dev and zlib-dev), I'd say it makes sense to place xz-dev on the same footing, given that it's the default compressor. So
xz-dev and --with-liblzma.

To handle source .dsc packages, the dpkg-source program is used, and
that one requires the command-line compression tools (xz, gzip, bzip2,
lzma).

You might want to have a peek at the debian/control file for what are
the current hard dependencies there. And I'm happy to clarify any that
might not seem obvious.

* The dpkg-dev might need a depends on dpkg.

Could you elaborate why this is necessary?

When building packages some of the dev tools require (AFAIR) at least
the following run-time tools:

* dpkg: to get the build architecture by dpkg-architecture and
anything calling it or using the Dpkg::Arch modules. While the call
to dpkg might eventually disappear, this would be switched to parse
a file shipped by the dpkg (run-time) package anyway.
* dpkg-deb: by dpkg-name and dpkg-scanpackages to extract information
from .deb archives.
* dpkg-query: to locate what package contains a specific shared
library, but then I guess this is a bit pointless if there's not
going to be anything installed in the dpkg db anyway.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)