XPost: linux.debian.bugs.dist

Package: debian-policy
Version: 4.1.0.0
Severity: wishlist

Hi,
when upstream tarballs need to be repacked because they contain non-dfsg
free data appending '+dfsg' to the upstream version seems common
practice. However some packages append '.dfsg', others use
+dfsg<number> and there are more formats around.

It would be great if policy could recommend +dfsg as the default
(falling back to +dfsg<number> if there was a packaging error (i.e. some
files slip through and the package was rejected by ftp-masters).

This would make it simpler for tools like lintian or gbp to detect repacked tarballs (in this case we don't want to attach the upstream signature to
the chages file).
Cheers,
-- Guido

-- System Information:
Debian Release: buster/sid
APT prefers testing
APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (500, 'oldoldstable'), (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.12.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages debian-policy depends on:
ii libjs-sphinxdoc 1.5.6-2

debian-policy recommends no packages.

Versions of packages debian-policy suggests:
pn doc-base <none>

-- no debconf information

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.bugs.dist

On Fri, 08 Sep 2017 at 10:38:54 -1000, David Prévot wrote:

Le 08/09/2017 à 07:44, Simon McVittie a écrit :

This made me think that we should maybe only be doing this when
a *pre-existing* upstream version needs to be repacked.
[…] when upstream releases
foo/1.2.4, even if the non-freeness has not been fixed, the
maintainer would repack it as 1.2.4 rather than 1.2.4+dfsg.

That is be a bit misleading: since we are not using the upstream
version, using the same version is a lie.

In general we are not using the upstream version anyway, and that's why
we append a Debian revision. The practical difference between, for
example, yquake2 7.01 and 7.01~dfsg1-2 is a lot smaller than the
difference between, for example, openssh 7.5p1 and openssh 7.5p1-5 (the
latest on sources.debian.net); so why does yquake2 need a special marker
that openssh does not?

(yquake2 drops Windows and Mac support files that would not affect the
Linux binary anyway, but its only patches are minor and to the Makefile; openssh patches the code significantly, adding large new features.)

Regards,
smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.bugs.dist

On Fri, 08 Sep 2017 at 16:10:44 +0200, Guido Günther wrote:

when upstream tarballs need to be repacked because they contain non-dfsg
free data appending '+dfsg' to the upstream version seems common
practice. However some packages append '.dfsg', others use
+dfsg<number> and there are more formats around.

It's a coincidence that you should mention this today. I've just run
into a situation where routinely appending +dfsg causes brokenness:

* upstream releases iortcw 1.51
* I package iortcw 1.51+dfsg1-1
* upstream releases iortcw 1.51b
* I package iortcw 1.51b+dfsg1-1
* oops! 1.51b+dfsg1 << 1.51+dfsg1, because b << +
* workaround: I had to relabel upstream's release as 1.51.b

This made me think that we should maybe only be doing this when
a *pre-existing* upstream version needs to be repacked. For example,
if foo/1.2.3 is found to be non-free after it is already in the
archive, the maintainer would repack it as 1.2.3+dfsg to get a new
orig.tar for the same upstream version; but when upstream releases
foo/1.2.4, even if the non-freeness has not been fixed, the
maintainer would repack it as 1.2.4 rather than 1.2.4+dfsg.

That would make +dsfg into a temporary hack a bit like the +really (anti-)pattern, rather than something long-lived that is routinely
applied to upstreams whose release tarballs are not entirely Free
(ioquake3 and its non-commercial bytecode compiler, gcc/make and
their non-DFSG docs, anything that bundles RFCs, etc.).

I notice that make and gcc don't routinely decorate their version numbers
in this way, and perhaps they're right to not do so.

This would make it simpler for tools like lintian or gbp to detect repacked tarballs (in this case we don't want to attach the upstream signature to
the chages file).

Why would the maintainer of such a package want to (arrange for uscan
to) rename the signature to foo_1.2.3.orig.tar.gz.asc where it would be
picked up by gbp? The signature is never going to validate successfully on
the orig tarball, so there is no point in doing that rename.

For repacked packages, if uscan is used at all, it would seem best for debian/watch to instruct uscan to download the signature, use it to
verify the non-repacked tarball, repack the tarball excluding problematic files, and *not* rename the signature to the name that contains .orig.
gbp won't see it, and is happy.

Lintian can detect that the tarball was repacked by looking inside
at the first few tar members - a repacked tarball is meant to
contain only a foo_1.2.3.orig/ directory (devref §6.7.8.2.4) and
non-repacked tarballs are unlikely to do so. In this case, ignoring debian/upstream/signing-key.asc for the purposes of deciding whether there ought to be an upstream signature would seem like a good Lintian
feature request?

S

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.bugs.dist

Hi,
On Fri, Sep 08, 2017 at 06:44:03PM +0100, Simon McVittie wrote:

On Fri, 08 Sep 2017 at 16:10:44 +0200, Guido Günther wrote:

when upstream tarballs need to be repacked because they contain non-dfsg free data appending '+dfsg' to the upstream version seems common
practice. However some packages append '.dfsg', others use
+dfsg<number> and there are more formats around.

It's a coincidence that you should mention this today. I've just run
into a situation where routinely appending +dfsg causes brokenness:

* upstream releases iortcw 1.51
* I package iortcw 1.51+dfsg1-1
* upstream releases iortcw 1.51b
* I package iortcw 1.51b+dfsg1-1
* oops! 1.51b+dfsg1 << 1.51+dfsg1, because b << +
* workaround: I had to relabel upstream's release as 1.51.b

This made me think that we should maybe only be doing this when
a *pre-existing* upstream version needs to be repacked. For example,
if foo/1.2.3 is found to be non-free after it is already in the
archive, the maintainer would repack it as 1.2.3+dfsg to get a new
orig.tar for the same upstream version; but when upstream releases
foo/1.2.4, even if the non-freeness has not been fixed, the
maintainer would repack it as 1.2.4 rather than 1.2.4+dfsg.

That would make +dsfg into a temporary hack a bit like the +really (anti-)pattern, rather than something long-lived that is routinely
applied to upstreams whose release tarballs are not entirely Free
(ioquake3 and its non-commercial bytecode compiler, gcc/make and
their non-DFSG docs, anything that bundles RFCs, etc.).

I notice that make and gcc don't routinely decorate their version numbers
in this way, and perhaps they're right to not do so.

This would make it simpler for tools like lintian or gbp to detect repacked tarballs (in this case we don't want to attach the upstream signature to the chages file).

Why would the maintainer of such a package want to (arrange for uscan
to) rename the signature to foo_1.2.3.orig.tar.gz.asc where it would be picked up by gbp? The signature is never going to validate successfully on the orig tarball, so there is no point in doing that rename.

That's basically the point: make it simple for people and tools to
figure out that it is a repacked tarball and therefore e.g. the
signature doesn't need to be attached. It also makes it simple for other
tools down the chain that there shouldn't be a signature (e.g. lintian
which warns about missing upstream signatures doesn't do so if it finds
+dfsg in the tarball name). Another reason is that people might want to
grab the upstream tarball when examining things (e.g. security issues)
since the Debian tarball isn't complete.

For repacked packages, if uscan is used at all, it would seem best for debian/watch to instruct uscan to download the signature, use it to
verify the non-repacked tarball, repack the tarball excluding problematic files, and *not* rename the signature to the name that contains .orig.
gbp won't see it, and is happy.

That's ok for uscan -> gbp but not for e.g. lintian checking later if
there should be a signature.

Lintian can detect that the tarball was repacked by looking inside
at the first few tar members - a repacked tarball is meant to
contain only a foo_1.2.3.orig/ directory (devref §6.7.8.2.4) and
non-repacked tarballs are unlikely to do so. In this case, ignoring debian/upstream/signing-key.asc for the purposes of deciding whether there ought to be an upstream signature would seem like a good Lintian
feature request?

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871957#36

Cheers,
-- Guido

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)