1. Forum
  2. Usenet
  3. LINUX.DEBIAN.DEVEL.QA

  • analyse-sbuild-log

    From Jelmer =?utf-8?Q?Vernoo=C4=B3?=@21:1/5 to All on Mon Apr 5 16:30:02 2021
    The python3-buildlog-consultant package now has a "analyse-sbuild-log"
    command that can parse and analyse sbuild logs and in most cases
    identity the line or lines that explain why a build failed.

    E.g.:

    % analyse-sbuild-log --json ~/build.log
    {"stage": "build", "section": "Build", "lineno": 857, "kind": "missing-python-module", "details": {"module": "setuptools", "python_version": 3, "minimum_version": null}}

    This is what powers the line selection for the fragments shown by the
    Debian janitor for failed builds.

    Cheers,

    Jelmer

    --
    Jelmer Vernooij <jelmer@jelmer.uk>
    PGP Key: https://www.jelmer.uk/D729A457.asc

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEsjhixBXWVlpOhsvXV5wWDUyeI+gFAmBrG00ACgkQV5wWDUye I+jGPw/+NTLwpmw0oSCQlaUklRcpkOdUz0CGjnHwRQzgGPSIXrv6pK8jjf6txutq YoqZNy/Kq/BQ4ze5y3iUAgBAhoKb9D9Gy4n4XTVHPAwt0+kF8ibqzTQYtT9Nfrep hzyKTS+kD4GCalSdXseIYBKuaYyBPhEs6S1SOg7Vw9c0aClKrv1Mch8lkKiwO3bu O3oTJ2q8dTBqS3LisOHbQxALTnzwiRwffCVlURGicjsfeUrWvppv9z+H+PDJSe0s IuTc+NIl5sWmnAWTfGArqBGqurf6ILcP4nE9u66bLt9ag4yVC8XwQjbUn/8bNowe ZaMjM4foAjU9KLTtAJJJtEq2h1pPq1Nf7n3zGPXrgMJbWDEjMsS/uTDO/+XbHMRE FX/y+1LFYK+6knX3/YhDdF+NyilRBJhIAkh1bWG/0BNvtrkUqMZExxl3da8pjcaQ NAWMP6Md/o0B4PuIrdWeRf27dhLBU/vF6+KCJKLlx6skRSWrIlCW8KzXRABRcJwq 8TV97vL38PpO/j9SoD3a7Rp1T3IPYWP1V3ejsj4SiF67Cqnho2WxEmyk30/deCRd Ql9vb5SYBpbFMLKn1t+4yqLWTbmOu3ZoWlz4o2UL9sr8YlRDX0qaVdfQR7nf2gbv mnSgD55xR+g1laZwTmbcHWg+J8B/oL/VZ7POi4+TYcdo7Zgg2IM=
    =Tc7x
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lucas Nussbaum@21:1/5 to All on Tue Apr 6 09:00:01 2021
    Hi,

    On 05/04/21 at 15:14 +0100, Jelmer Vernooij wrote:
    The python3-buildlog-consultant package now has a "analyse-sbuild-log" command that can parse and analyse sbuild logs and in most cases
    identity the line or lines that explain why a build failed.

    E.g.:

    % analyse-sbuild-log --json ~/build.log
    {"stage": "build", "section": "Build", "lineno": 857, "kind": "missing-python-module", "details": {"module": "setuptools", "python_version": 3, "minimum_version": null}}

    This is what powers the line selection for the fragments shown by the
    Debian janitor for failed builds.

    Nice!
    I've done something similar for my archive rebuild scripts[1]. Maybe we
    should try merge efforts in some way?

    [1] see log-parser-* in https://salsa.debian.org/lucas/collab-qa-tools/-/tree/master/lib/collab-qa
    test cases are https://salsa.debian.org/lucas/collab-qa-tools/-/tree/master/test

    Some comments based on my own experience:
    - did you consider adding a mode where it would output log lines that
    precede the failure from the last interesting point, to make it easy to
    file bugs?
    - running your tool against my test cases would be interesting

    Lucas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jelmer =?utf-8?Q?Vernoo=C4=B3?=@21:1/5 to Lucas Nussbaum on Tue Apr 6 11:20:02 2021
    On Tue, Apr 06, 2021 at 08:49:01AM +0200, Lucas Nussbaum wrote:
    Hi,

    On 05/04/21 at 15:14 +0100, Jelmer Vernooij wrote:
    The python3-buildlog-consultant package now has a "analyse-sbuild-log" command that can parse and analyse sbuild logs and in most cases
    identity the line or lines that explain why a build failed.

    E.g.:

    % analyse-sbuild-log --json ~/build.log
    {"stage": "build", "section": "Build", "lineno": 857, "kind": "missing-python-module", "details": {"module": "setuptools", "python_version": 3, "minimum_version": null}}

    This is what powers the line selection for the fragments shown by the Debian janitor for failed builds.

    Nice!
    I've done something similar for my archive rebuild scripts[1]. Maybe we should try merge efforts in some way?
    Perhaps we can share a common data file with the single-line regexes?
    That's where the bulk of my effort is today, and that seems like the
    easiest way to share efforts.

    [1] see log-parser-* in https://salsa.debian.org/lucas/collab-qa-tools/-/tree/master/lib/collab-qa
    test cases are https://salsa.debian.org/lucas/collab-qa-tools/-/tree/master/test

    Some comments based on my own experience:
    - did you consider adding a mode where it would output log lines that
    precede the failure from the last interesting point, to make it easy to
    file bugs?
    Yep, it has a context mode (similar to diff) as of yesterday.

    - running your tool against my test cases would be interesting
    It seems to identify the problematic line in almost (all?) the same
    cases, but doesn't always provide a better classification than your
    script does - i.e. it finds python exceptions and segmentation faults
    but doesn't call them out as such.

    Cheers,

    Jelmer

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEsjhixBXWVlpOhsvXV5wWDUyeI+gFAmBsJY0ACgkQV5wWDUye I+gSixAArryF6U/evESgPUVsDkOVNozt/7SM37DWQHArefhq+wYnOiZ7a3bspu/c TgROtTQhtxhkkVZ/rjXrl6gv6siRakQFiaYrkjbhZjFEp4458D75dg6tnynTtGuc m/yrgc4qZwe9kTYkAT86ehCE1p9A0tFuOW4OQR54Bj2rl6dkXikuZyYQ7UU9Pna6 T8SlckyZOlxU2xvn6FcSvSWK7FNu36aV8mPheGZM39/dzbGHiiw6vXx5+2lzCLpJ jvk+ykIMguy72CEnmYysFbcQlv/DoP1+C+m4idixTzM7cNOYVt7Qv7HIcN/A+suP BUBly2fVHnnxOXHc+1Yi2tCdT+n6TnwWBRo0ghPjmqC/FzHer6kOhoi7zkeOSDs4 3wIs7IvdsXB2vQ/Gk2w39UWu4Da5P8MonfYLe9DbOs/5maTlArYcT/ca+ZoLzrx6 djrc6fJTx0CrcQJgHKsL4JjRxHkmM33LbI+q2GfyQwIrujhVVIUBaqK73M9nnIi1 BpTczXtdUB8arUABfTfsPUznpilEFg+6+D7FyFW5xJKy3j6LH02Uc5LVhhkL3/hN 8wz1sP6bFWxRDGgwhJDXrM8lcPo1dHkLUG11k5YIJnHW4O6upG3/G7MFowU37mc2 7D3mHV++zJxij6vQa4F4FWTQyHQSKMjqCoQbswHu+75zkbopjJg=
    =EUZQ
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)