• Re: Problems verifying signed github releases (Re: Q: uscan with GitHub

    From FC Stegerman@21:1/5 to All on Sun Feb 19 21:10:01 2023
    * Guillem Jover <guillem@debian.org> [2023-02-19 20:50]:
    My upstream creates a tarball with git-archive, creates a signature and uploads it (as described in the wiki[3]). This used to work to verify
    the github-created tarball, but fails now - while creating my own
    tarball like upstream and verifying it with upstream's signature works.

    The uncompressed .tar files are identical (same hashsum), just the tar.gz differ. Does anyone know why, and how to fix it? I tried non-default compression levels for gzip with git-archive, but that didn't help to get an
    identical tar.gz like the one from github.

    I'd like to avoid having my upstream downloading the github-created tarball, verify&sign it and then upload this signature.

    I assume you (or whatever service or tool is failing the verification
    while creating a local tarball) might be seeing issues with git having switched implementation for gzip, and a mismatch with the implementation being used in either side. Perhaps try to set git's
    tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use
    the external command instead of the internal implementation? Or perhaps
    you are using an old git that defaults to the external gzip but upstream
    uses the internal one?

    I was going to suggest that might be the issue, but you were faster :)
    I do have some relevant links:

    https://reproducible-builds.org/reports/2023-01/#news https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002709.html
    https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002710.html

    (There was a recent LWN article covering this, see https://lwn.net/Articles/921787/.)

    That seems to be subscribers-only :(

    - FC

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Guillem Jover@21:1/5 to Jens Reyer on Sun Feb 19 21:00:01 2023
    Hi!

    On Sun, 2023-02-19 at 18:34:56 +0100, Jens Reyer wrote:
    [This is a followup to the thread "Q: uscan with GitHub" at https://lists.debian.org/debian-devel/2022/10/msg00313.html. I manually set in-reply-to, but am not sure if it'll work.]

    My upstream creates a tarball with git-archive, creates a signature and uploads it (as described in the wiki[3]). This used to work to verify
    the github-created tarball, but fails now - while creating my own
    tarball like upstream and verifying it with upstream's signature works.

    The uncompressed .tar files are identical (same hashsum), just the tar.gz differ. Does anyone know why, and how to fix it? I tried non-default compression levels for gzip with git-archive, but that didn't help to get an identical tar.gz like the one from github.

    I'd like to avoid having my upstream downloading the github-created
    tarball, verify&sign it and then upload this signature.

    I assume you (or whatever service or tool is failing the verification
    while creating a local tarball) might be seeing issues with git having
    switched implementation for gzip, and a mismatch with the implementation
    being used in either side. Perhaps try to set git's
    tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use
    the external command instead of the internal implementation? Or perhaps
    you are using an old git that defaults to the external gzip but upstream
    uses the internal one?

    (There was a recent LWN article covering this, see https://lwn.net/Articles/921787/.)

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jens Reyer@21:1/5 to FC Stegerman on Mon Feb 20 01:00:01 2023
    On 19.02.23 21:08, FC Stegerman wrote:
    * Guillem Jover <guillem@debian.org> [2023-02-19 20:50]:
    My upstream creates a tarball with git-archive, creates a signature and
    uploads it (as described in the wiki[3]). This used to work to verify
    the github-created tarball, but fails now - while creating my own
    tarball like upstream and verifying it with upstream's signature works.

    The uncompressed .tar files are identical (same hashsum), just the tar.gz >>> differ. Does anyone know why, and how to fix it? I tried non-default
    compression levels for gzip with git-archive, but that didn't help to get an
    identical tar.gz like the one from github.

    I'd like to avoid having my upstream downloading the github-created
    tarball, verify&sign it and then upload this signature.

    I assume you (or whatever service or tool is failing the verification
    while creating a local tarball) might be seeing issues with git having
    switched implementation for gzip, and a mismatch with the implementation
    being used in either side. Perhaps try to set git's
    tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use >> the external command instead of the internal implementation? Or perhaps
    you are using an old git that defaults to the external gzip but upstream
    uses the internal one?

    I was going to suggest that might be the issue, but you were faster :)
    I do have some relevant links:

    https://reproducible-builds.org/reports/2023-01/#news https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002709.html
    https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002710.html

    (There was a recent LWN article covering this, see
    https://lwn.net/Articles/921787/.)

    That seems to be subscribers-only :(

    - FC

    Thank you both, that solved it!

    After further digging into the details I updated the wiki to suggest:

    git -c tar.tar.gz.command='gzip -cn' \
    archive --format=tar.gz --prefix="${tag}/" \
    -o "../${tag}.tar.gz" "${tag}"

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stephan =?ISO-8859-1?Q?Verb=FCcheln@21:1/5 to All on Mon Feb 20 08:10:01 2023
    Note that kernel.org signs the raw tar file and not the compressed
    file. This way, they avoid issues like that and also allow conversion
    into different compression formats while the signature stays valid.

    Downside is that you have to decompress it first and then hash quite a
    big file for validation.

    Regards

    -----BEGIN PGP SIGNATURE-----

    iIwEABYIADQWIQRB1rjSpCJd8a7h6mNgNUJZCjx8YgUCY/MauhYcdmVyYnVlY2hl bG5AcG9zdGVvLmRlAAoJEGA1QlkKPHxidTYA+wWd09jHbBwwJpSi+1D9i0XWVItm NPX+xjg2bBU7CLdSAP9hgRR+1Qsza4czgzQOZh22FFTZ1NwYASop+IOjULgOBg==
    =zge9
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakub Wilk@21:1/5 to All on Mon Feb 20 07:20:01 2023
    * FC Stegerman <flx@obfusk.net>, 2023-02-19 21:08:
    (There was a recent LWN article covering this, see >>https://lwn.net/Articles/921787/.)

    That seems to be subscribers-only :(

    Here you go:
    https://lwn.net/SubscriberLink/921787/ff1350f40f12fb8e/

    --
    Jakub Wilk

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)