• debsig-verify test data contains signatures w/o debian-binary included

    From Charles Duffy@21:1/5 to All on Wed May 5 20:10:02 2021
    Howdy --

    I notice that the test/debs/ source tree in debsig-verify includes files
    made prior to the 0.5 release, at which time the "debian-binary" file was
    not expected to be concatenated into the other data to be signed.

    Compare:

    [chaduffy@shiny5l:~/VC/debsig-verify/test/debs]$ gpg --verify <(ar p sigtest2_2.0-1_all.deb _gpgorigin) <(ar p sigtest2_2.0-1_all.deb
    control.tar.gz data.tar.gz)
    gpg: Signature made Mon 04 Dec 2000 04:13:50 PM CST
    gpg: using DSA key 0x7CD73F641E04EC2D
    gpg: Good signature from "Ben Collins <bcollins@linux.com>" [unknown]
    gpg: aka "Ben Collins <bcollins@debian.org>" [unknown]
    gpg: aka "Ben Collins <bcollins@openldap.org>" [unknown]
    gpg: aka "Ben Collins <bmc@visi.net>" [unknown]
    gpg: aka "Ben Collins <collinbm@djj.state.va.us>" [unknown] gpg: aka "Ben Collins (Nada) <bcollins@debian.org>"
    [unknown]
    gpg: WARNING: This key is not certified with a trusted signature!
    gpg: There is no indication that the signature belongs to the
    owner.
    Primary key fingerprint: CA9C 9B60 D31F B7FE B093 CDA1 7CD7 3F64 1E04 EC2D

    [chaduffy@shiny5l:~/VC/debsig-verify/test/debs]$ gpg --verify <(ar p sigtest2_2.0-1_all.deb _gpgorigin) <(ar p sigtest2_2.0-1_all.deb
    debian-binary control.tar.gz data.tar.gz)
    gpg: Signature made Mon 04 Dec 2000 04:13:50 PM CST
    gpg: using DSA key 0x7CD73F641E04EC2D
    gpg: BAD signature from "Ben Collins <bcollins@linux.com>" [unknown]

    ...whereas with a modern .deb file recently signed by the debsigs tool, the debian-binary file _must_ be concatenated into the other data (reference
    its source at https://gitlab.com/debsigs/debsigs/-/blob/52cb60820a1d65b87d3818d3e12945f27e88c57d/debsigs#L94
    ).

    It would be frankly less confusing to remove test data that's no longer relevant, though replacing it with a valid test would be better (or
    including sample data in both formats if there's backwards compatibility
    logic to be tested, though I see no such logic in the codebase).

    This came up in the context of having borrowed the test data from
    debsig-verify to use in a Go reimplementation at https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go to ensure compatibility; it turns out that using this test data ensured that the
    result would _not_ be compatible with the modern format!

    <div dir="ltr">Howdy --<div><br></div><div>I notice that the test/debs/ source tree in debsig-verify includes files made prior to the 0.5 release, at which time the &quot;debian-binary&quot; file was not expected to be concatenated into the other data to
    be signed.</div><div><br></div><div>Compare:</div><div><br></div><div>[chaduffy@shiny5l:~/VC/debsig-verify/test/debs]$ gpg --verify &lt;(ar p sigtest2_2.0-1_all.deb _gpgorigin) &lt;(ar p sigtest2_2.0-1_all.deb control.tar.gz data.tar.gz)<br>gpg:
    Signature made Mon 04 Dec 2000 04:13:50 PM CST<br>gpg:                using DSA key 0x7CD73F641E04EC2D<br>gpg: Good signature from &quot;Ben Collins &lt;<a href="mailto:bcollins@linux.com">bcollins@linux.com</a>&gt;&quot; [unknown]<br>gpg:     Â
                aka &quot;Ben Collins &lt;<a href="mailto:bcollins@debian.org">bcollins@debian.org</a>&gt;&quot; [unknown]<br>gpg:                 aka &quot;Ben Collins &lt;<a href="mailto:bcollins@openldap.org">bcollins@openldap.org</a>&gt;&
    quot; [unknown]<br>gpg:                 aka &quot;Ben Collins &lt;<a href="mailto:bmc@visi.net">bmc@visi.net</a>&gt;&quot; [unknown]<br>gpg:                 aka &quot;Ben Collins &lt;<a href="mailto:collinbm@djj.state.va.us">collinbm@djj.
    state.va.us</a>&gt;&quot; [unknown]<br>gpg:                 aka &quot;Ben Collins (Nada) &lt;<a href="mailto:bcollins@debian.org">bcollins@debian.org</a>&gt;&quot; [unknown]<br>gpg: WARNING: This key is not certified with a trusted signature!<br>
    gpg:          There is no indication that the signature belongs to the owner.<br>Primary key fingerprint: CA9C 9B60 D31F B7FE B093  CDA1 7CD7 3F64 1E04 EC2D<br><br>[chaduffy@shiny5l:~/VC/debsig-verify/test/debs]$ gpg --verify &lt;(ar p sigtest2_2.0-
    1_all.deb _gpgorigin) &lt;(ar p sigtest2_2.0-1_all.deb debian-binary control.tar.gz data.tar.gz)<br>gpg: Signature made Mon 04 Dec 2000 04:13:50 PM CST<br>gpg:                using DSA key 0x7CD73F641E04EC2D<br>gpg: BAD signature from &quot;Ben
    Collins &lt;<a href="mailto:bcollins@linux.com">bcollins@linux.com</a>&gt;&quot; [unknown]<br></div><div><br></div><div>...whereas with a modern .deb file recently signed by the debsigs tool, the debian-binary file _must_ be concatenated into the other
    data (reference its source at <a href="https://gitlab.com/debsigs/debsigs/-/blob/52cb60820a1d65b87d3818d3e12945f27e88c57d/debsigs#L94">https://gitlab.com/debsigs/debsigs/-/blob/52cb60820a1d65b87d3818d3e12945f27e88c57d/debsigs#L94</a>).</div><div><br></
    <div>It would be frankly less confusing to remove test data that&#39;s no longer relevant, though replacing it with a valid test would be better (or including sample data in both formats if there&#39;s backwards compatibility logic to be tested,
    though I see no such logic in the codebase).</div><div><br></div><div>This came up in the context of having borrowed the test data from debsig-verify to use in a Go reimplementation at <a href="https://github.com/paultag/go-debian/blob/master/deb/
    sigcheck.go">https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go</a> to ensure compatibility; it turns out that using this test data ensured that the result would _not_ be compatible with the modern format!</div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Guillem Jover@21:1/5 to Charles Duffy on Thu May 6 04:40:01 2021
    Hi!

    On Wed, 2021-05-05 at 12:51:18 -0500, Charles Duffy wrote:
    I notice that the test/debs/ source tree in debsig-verify includes files
    made prior to the 0.5 release, at which time the "debian-binary" file was
    not expected to be concatenated into the other data to be signed.

    […]

    ...whereas with a modern .deb file recently signed by the debsigs tool, the debian-binary file _must_ be concatenated into the other data (reference
    its source at https://gitlab.com/debsigs/debsigs/-/blob/52cb60820a1d65b87d3818d3e12945f27e88c57d/debsigs#L94
    ).

    It would be frankly less confusing to remove test data that's no longer relevant, though replacing it with a valid test would be better (or
    including sample data in both formats if there's backwards compatibility logic to be tested, though I see no such logic in the codebase).

    Ah, thanks. I think I did not remove this when I took over, because I
    probably wanted to preserve the policies usage at least as examples or
    to base new test cases on those or similar. But you are completely
    correct that having this laying around creates more confusion than the potential benefit of keeping them, so I've just removed them.

    This came up in the context of having borrowed the test data from debsig-verify to use in a Go reimplementation at https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go to ensure compatibility; it turns out that using this test data ensured that the
    result would _not_ be compatible with the modern format!

    Hmm, it seems though that implementation is not compliant, as it
    hardcodes several types, and does not use the defined policies.

    In any case I'm not sure how useful is to reimplement this now, as
    the debsigs infra needs to be revamped to be able to integrate it
    properly into dpkg and DAK, mainly how the signatures are stored in
    the .deb. I've also started pondering about switching the policy
    from XML to JSON, and started some code on that direction.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Charles Duffy@21:1/5 to guillem@debian.org on Thu May 6 17:40:01 2021
    On Wed, May 5, 2021 at 9:33 PM Guillem Jover <guillem@debian.org> wrote:

    On Wed, 2021-05-05 at 12:51:18 -0500, Charles Duffy wrote:
    This came up in the context of having borrowed the test data from debsig-verify to use in a Go reimplementation at https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go to
    ensure
    compatibility; it turns out that using this test data ensured that the result would _not_ be compatible with the modern format!

    Hmm, it seems though that implementation is not compliant, as it
    hardcodes several types, and does not use the defined policies.


    I'm sorry -- by using the word "reimplementation" I may have implied a goal
    to be feature-complete or configuration-compatible or otherwise to
    eventually serve as a replacement for the original project, and this is not
    the case.

    Whereas upstream debsig-verify has policy-based configuration, my goal in writing the extension for Paul's Go library referenced above was purely to implement the actual signature checking itself against a keyring provided
    by the library user, leaving policies used to decide which keys should be
    in the keyring used when validating a given package out-of-scope and to be implemented by users of that library. Implementing parsing of the policy
    format used by debsig-verify doesn't strike me as an utterly inappropriate addition to go-debian (though of course Paul has final authority on that)
    -- but it is something where, were I implementing it myself, I'd want it to
    be a separate module, not tightly coupled to the code that does the actual validation of each package, with each component usable on its own; this
    would make it easy to implement additional or alternate policy engines independent of alternate package-level signature formats.

    Within the limited scope that the current Go implementation aims for, I'm
    very interested in knowing where it could be improved, if you'd be willing
    to go into more detail.

    In any case I'm not sure how useful is to reimplement this now, as
    the debsigs infra needs to be revamped to be able to integrate it
    properly into dpkg and DAK, mainly how the signatures are stored in
    the .deb. I've also started pondering about switching the policy
    from XML to JSON, and started some code on that direction.


    I look forward to extending the Go library to cover new format versions as appropriate in the future.

    <div dir="ltr"><div dir="ltr">On Wed, May 5, 2021 at 9:33 PM Guillem Jover &lt;<a href="mailto:guillem@debian.org">guillem@debian.org</a>&gt; wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-
    left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, 2021-05-05 at 12:51:18 -0500, Charles Duffy wrote:<br>&gt; This came up in the context of having borrowed the test data from<br>
    &gt; debsig-verify to use in a Go reimplementation at<br>
    &gt; <a href="https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go" rel="noreferrer" target="_blank">https://github.com/paultag/go-debian/blob/master/deb/sigcheck.go</a> to ensure<br>
    &gt; compatibility; it turns out that using this test data ensured that the<br> &gt; result would _not_ be compatible with the modern format!<br>

    Hmm, it seems though that implementation is not compliant, as it<br>
    hardcodes several types, and does not use the defined policies.<br></blockquote><div><br></div><div>I&#39;m sorry -- by using the word &quot;reimplementation&quot; I may have implied a goal to be feature-complete or configuration-compatible or otherwise
    to eventually serve as a replacement for the original project, and this is not the case.</div><div><br></div><div>Whereas upstream debsig-verify has policy-based configuration, my goal in writing the extension for Paul&#39;s Go library referenced above
    was purely to implement the actual signature checking itself against a keyring provided by the library user, leaving policies used to decide which keys should be in the keyring used when validating a given package out-of-scope and to be implemented by
    users of that library. Implementing parsing of the policy format used by debsig-verify doesn&#39;t strike me as an utterly inappropriate addition to go-debian (though of course Paul has final authority on that) -- but it is something where, were I
    implementing it myself, I&#39;d want it to be a separate module, not tightly coupled to the code that does the actual validation of each package, with each component usable on its own; this would make it easy to implement additional or alternate policy
    engines independent of alternate package-level signature formats.</div><div><br></div><div>Within the limited scope that the current Go implementation aims for, I&#39;m very interested in knowing where it could be improved, if you&#39;d be willing to go
    into more detail.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
    In any case I&#39;m not sure how useful is to reimplement this now, as<br>
    the debsigs infra needs to be revamped to be able to integrate it<br>
    properly into dpkg and DAK, mainly how the signatures are stored in<br>
    the .deb. I&#39;ve also started pondering about switching the policy<br>
    from XML to JSON, and started some code on that direction.<br></blockquote><div><br></div><div>I look forward to extending the Go library to cover new format versions as appropriate in the future. </div></div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)