• pyyaml 6

    From Gordon Ball@21:1/5 to All on Fri Oct 7 00:30:01 2022
    pyyaml (aka python3-yaml) is an rdepend for >300 packages. We currently
    have 5.4.1, but version 6 was released late last year, which does quite
    a lot of cleanup (eg, dropping python 2 support) and disables unsafe
    loading (arbitrary python code execution) unless explicitly opted into.

    Unfortunately this is a breaking change for any code which uses the
    simple `yaml.load(fileobj)` idiom - it now needs to be `yaml.safe_load(fileobj)` or `yaml.load(fileobj, Loader=yaml.SafeLoader)`.

    I uploaded 6.0 to experimental late last year, but time constraints
    meant it never got pushed forward. I've recently refreshed the version
    in experimental (experimental doesn't get binnmus so it was still only
    built for 3.9), and there are relatively few autopkgtest regressions: https://qa.debian.org/excuses.php?experimental=1&package=pyyaml

    However, using codesearch suggests there are quite a few places which
    are likely to break: http://codesearch.debian.net/search?q=yaml%5B.%5Dload%5B%28%5D%5B%5E%2C%5D%2B%5B%29%5D+filetype%3Apython&literal=0&perpkg=1

    Some of these are false positives, such as invocations of ruamel.yaml,
    that string appearing in documentation, or things regex can't catch -
    but that still looks like it leaves a significant number of packages
    being potentially broken - and presumably lacking autopkgtest coverage).


    So, I'd seek some input on how to move forward.

    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for
    an upload?
    * Wait until after the freeze?

    The only bug requesting it actually be upgraded is https://bugs.debian.org/1008262 (for openstack). I don't know if that
    has proved a hard blocker - I _think_ anything designed to work with 6.x
    should also work with 5.4.


    Gordon

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Timo =?utf-8?Q?R=C3=B6hling?=@21:1/5 to All on Fri Oct 7 01:20:01 2022
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for
    an upload?
    * Wait until after the freeze?
    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks
    is probably in dire need of maintenance anyway.


    Cheers
    Timo

    --
    ⢀⣴⠾⠻⢶⣦⠀ ╭────────────────────────────────────────────────────╮
    ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │
    ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │
    ⠈⠳⣄⠀⠀⠀⠀ ╰────────────────────────────────────────────────────╯

    -----BEGIN PGP SIGNATURE-----

    iQGzBAEBCgAdFiEEJvtDgpxjkjCIVtam+C8H+466LVkFAmM/YPoACgkQ+C8H+466 LVmnagv/SgnmBNSsAk8LSCT7VdTHBCviYxE/HmdoCL8xmBlg5L5/gyp5W7IveX6K CHH3+3UFW2fuQLCh5MnbV01j1UMBCA5w+KwgMkxeBEVUksbMhy+irNf4hSYRngXx I3yRwzspU976ACs2OBZY5GNM8HLfKdaRhKw0L8CQhg9wPsE3lG0U4i5bjnIe6dHA HPuLD5bVPg2D9QYfmRJwPC3N/+C1m6+ymh7ORlC/BrcK83W5zKMNqQmf1rBbX6rt QcpmjglxZwUHNICtAlsNpIvKtqwZvXtwNqErxryqti0wXLykLw4l5JQAMUth6Q9V i8agahsmA8qgb2EMkLTXvHWQ/bLjr3T/0lg6k8rZGrS
  • From Paul Wise@21:1/5 to Gordon Ball on Fri Oct 7 03:50:01 2022
    On Fri, 2022-10-07 at 00:10 +0200, Gordon Ball wrote:

    * Upload to unstable and see what breaks?

    The experimental pseudo-excuses already say several packages break:

    https://qa.debian.org/excuses.php?experimental=1&package=pyyaml

    autopkgtest for ganeti/3.0.2-1: amd64: Regression, arm64: Regression
    autopkgtest for llvm-toolchain-13/1:13.0.1-7: amd64: Pass, arm64: Regression
    autopkgtest for satpy/0.37.1-1: amd64: Regression, arm64: Regression
    autopkgtest for spades/3.15.5+dfsg-1: amd64: Regression

    So at least these issues need to be investigated and maybe bugs filed.

    * Request an archive rebuild with this version and see what breaks?

    Definitely.

    * File bugs against all likely affected packages with a fixed date for
    an upload?

    Definitely.

    I don't know if any packages in Debian have versioned dependencies on
    pyyaml, but if they do then it might be worth filing a transition bug.
    Probably also a good idea to do that anyway too.

    https://wiki.debian.org/Teams/ReleaseTeam/Transitions

    There might also be Debian services broken by pyyaml 6, but they can be
    dealt with during the upgrade of the debian.org machines to bookworm.

    --
    bye,
    pabs

    https://wiki.debian.org/PaulWise

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmM/hF8ACgkQMRa6Xp/6 aaOxpBAAo+cDt6xE9zzatTZTuPgVw0W4XqYzFJzYEbMEBVDOTpt0j431Wl9RhBSe u3M8A7aIGTt5PMICAlIRsGqCSExA/LUkfnxnk8WPGSB2D9n3DodbmiXyiuAm24wK HQll+RIyyzuGhtEk0p5jMpY2kBIAOIi37QMHmDZazQOEragn1jUQn49gAs3UbO8M yl647kmp+zYVGg1L2OK4ah85Aap4jqXv9Lgl9er5bSVouHbR+xVajoDNEVwf9BSL KFWqgLbtUTvK+c+Sjb1Vrl57uZqBJbO9wWo0xoK3A1hUG4MQTt6xIhsNirbnljWE Z5y1tJBKGlYPoaLuZb/5qVCpqaSReabCsnhg1Wk44DC5HYtH62qOW8irgWvB1uZe +wp2p4E41QHh0Tzx0GTt3W0XfbQT9Z+IfUWHMhM2WUHUCTrd113FecWWz9gQdZl0 38CqAoQTY84LhUKkyU4HWGn4h1CGCyaOAc823jInL9fv1SxM9xDn3ctbA9XxyXIM fuZFfTjzWPV/PtNfad8YhhR5qfzzAgtzMzHhT2LmHw91hbQvOst5UTNBR3RBgBbc gf+aYH1JO8+RBqfCTs/trgTMqA19w4bGI1wKTspY7jMiF6/0SZKBcGGtioZm1WHj mVGiuFf/jOYyZ0js9rGXjPlhU3OsyFtpSjx8QCqMl32cHADXVBE=
    =Ex95
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeremy Stanley@21:1/5 to Gordon Ball on Fri Oct 7 16:50:01 2022
    On 2022-10-07 00:10:21 +0200 (+0200), Gordon Ball wrote:
    [...]
    The only bug requesting it actually be upgraded is https://bugs.debian.org/1008262 (for openstack). I don't know if
    that has proved a hard blocker - I _think_ anything designed to
    work with 6.x should also work with 5.4.

    I have a feeling 5.4 would also work for the latest versions of
    OpenStack Horizon. The change which increased the PyYAML minimum for
    it to >=6.0 didn't really say why it picked that minimum, other than
    for the sake of consistency:

    https://review.opendev.org/834053

    This can be tested upstream if folks think that would be a helpful
    data point, but it's not entirely trivial since I'll have to do some
    extra work to override the requirement constraints (otherwise I
    would have just done it before replying).
    --
    Jeremy Stanley

    -----BEGIN PGP SIGNATURE-----

    iQKTBAABCgB9FiEEl65Jb8At7J/DU7LnSPmWEUNJWCkFAmNAMt5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDk3 QUU0OTZGQzAyREVDOUZDMzUzQjJFNzQ4Rjk5NjExNDM0OTU4MjkACgkQSPmWEUNJ WCl5jA//V3X9jLDQ06jnfWxubVgJshrohn1Bt+XQFnWAv6mvEttepGutNRnOxbYZ MFAWIBl0av3QyVqBG66ZiGs4S+B5GDHbbSHpMwE2QBedafaDDVSDlQsUXlAeN/vR bbpRR7sKmud9FIcTLQ8F1JwrlsQfNX1LUQrP9RvVkGIHeXvNe3tcwCHBd7Unjp6G PO6MFH6fSyBqxLFG6OuAfCSIDLJ45IsJLVRc6YW5ZIxBwa0u1YRuHSnvFKYI6BvH HDyFLN4KHAOkP4Tlwa+uErG4yHncPfjbeNyCdEztQ1xrZ3sygr9YD7LFLLdjc3uu Q76Jl6lzwwZPYaxyRsAGc6I1iH8IPyPnv9AIR5jF6bLV/U5hlIFwjhpLu6K5roy8 CWn1+u7SEAc/HKAgGUQvQhoGLjV1hAMRmlvKeuIuI8NXNaJ21jweRHIV4ty1T1BZ h8ztsXQsGHBFyRmEjbBI/oai94qnwHnXu38HHmrjsWGjizA3jH3iirWvRrlpKema vZc125oj5D7LBSWaeO4/Aq8PGoExSzw1OtsqkU1K1LV96mBc8AoiFexo0gy838yv t0bgAFP2u1dSIy9dxUFVwygWW2SdamBjYKL38qJzs/2GLZQNNIBJv3v7CW+RhtId uhQtaC6hsxIrFudV7ldF7ZVYZpi96mu7jTgLeric/TYhmrcsY4A=
    =tgHZ
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32
  • From Jeremy Stanley@21:1/5 to Gordon Ball on Sun Oct 9 22:00:01 2022
    On 2022-10-09 21:39:56 +0200 (+0200), Gordon Ball wrote:
    [...]
    gnocchi # confirm, in gnocchi/gendoc

    Looks like it was fixed in gnocchi 4.4.2 earlier this year (unstable
    still has 4.4.0).

    jeepyb # confirm, in cmd/notify_impact

    I'm honestly surprised this is packaged for Debian, since it's just
    a pile of random scripts and hacks we use in OpenDev for things like
    Gerrit hooks, and that notify_impact tool has been unused for a long
    time. I could easily fix the yaml.load() call upstream, but I'm
    tempted to just go on a cleaning spree and remove a bunch of unused
    stuff like that from the repository.

    refstack-client # confirm, in refstack_client

    Fixed in 0.1.0 from February (unstable carries an unreleased
    snapshot from last year).
    --
    Jeremy Stanley

    -----BEGIN PGP SIGNATURE-----

    iQKTBAABCgB9FiEEl65Jb8At7J/DU7LnSPmWEUNJWCkFAmNDJ/NfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDk3 QUU0OTZGQzAyREVDOUZDMzUzQjJFNzQ4Rjk5NjExNDM0OTU4MjkACgkQSPmWEUNJ WCmojw/6AnvXefBsjgCIEo2zXyQDwyryn7n/mEdFCNgQHw1UPhZ5IjsztjmmmZrA z5aN3eOx208u6ZvXuc5fD28fNwN+jOEbtk9n89KFPPyvx1HDoZKNSOyDI/Clkz68 cumaFYlYRFNG/GquH4tED9OLYiL5oTvw9JmmUnxMgWHRVP0eUJO0T/VxBLZtVVc5 Wuyu+fpv4iCmD2/ItrOduv9BPa8u+NPR0wgKYEcWSZZj8KxCXSwT4qchEuSebRmb oUvOT6EVqrwYgHH+8hGEhBN/hpGIw7jTQYGnASQhrhaa3DPy/6wmidox3LmGrzeU Ti736zNlMZy4srYOXvJcLlLXRH5ZXvMD7xN6DXLSp4+f3xR+hg7ecMnkLV3iQfq9 xqNCMWduI4bfineM0kFUiQMc1+WFwQ1DlHqN+tGRZkU19g/ocS8mMgAiVMhxL7DW j1Ln3w/+LAvpyJ/lxoh/mdr4F5F8qPrHJA2qohrh0eibq/N1PlgEb23IwUitzPQh v7AWgaBmTrDvv7bA8oEjkzO6/xOAVeLKXNl/0W2NlPXVAQaD/fWZHdr+BrpPzGiK VlQX2s5qD0bfNuxR+QMdm1B83079s/v443qNEXKw7+IR/vos793uJgG/CW/2j2lk cfnIMt+itTyqmBbhmH67WK1Q7Qsb9C+qlwNw2JYfKCQMmmEzE9M=
    =Gn/D
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32
  • From Gordon Ball@21:1/5 to All on Sun Oct 9 21:50:01 2022
    On 07/10/2022 01:13, Timo Röhling wrote:
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for
    an upload?
    * Wait until after the freeze?
    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks
    is probably in dire need of maintenance anyway.

    Yes, that's a good point.

    I've done some more investigation of reverse-depends and looking for
    likely bad patterns with codesearch.d.n, and the set of packages which
    A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is
    pyyaml and B) actually depend or build-depend on `python3-yaml` is
    smaller than I feared. This is what I came up with:

    ```
    ansible # confirm - in ansible_collections/community/general/plugins
    buildbot # confirm - in porttostable
    ceph # confirm, in civetweb/third_party/duktape, src/test/crimson docker-compose # confirm, in tests
    dose3 # confirm, in scripts
    elasticsearch-curator # confirm, in test/unit
    etm # confirm, in etmTk/data
    ganeti # confirm, in qa, test/py
    gnocchi # confirm, in gnocchi/gendoc
    gr-dab # confirm, in python/app
    jeepyb # confirm, in cmd/notify_impact
    knitpy # confirm, in knitpy
    labgrid # confirm, in remote/coordinator
    lecm # confirm, in lecm/configuration
    lirc # confirm, in lirc/database
    lqa # confirm, in lqa_api/job
    open-adventure # confirm, in tests
    owslib # confirm, in ogcapi
    policyd-rate-limit # confirm, in config, tests
    python-aptly # confirm, in publisher
    python-canmatrix # confirm, in canmatrix/formats/yaml
    python-multipart # confirm, in multipart/tests
    python-pybedtools # confirm, in test, contrib
    python-seqcluster # confirm, in install
    python-tempestconf # confirm, in config_tempest/profile
    qcat # confirm, in adapters
    qcengine # confirm, in config
    refstack-client # confirm, in refstack_client
    relatorio # confirm, in templates/chart
    ripe-atlas-tools # confirm, in tools/settings
    ros-genpy # confirm, in test
    ros-rosinstall # confirm, in test, src/rosinstall
    ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
    spades # confirm, in assembler/src/test/teamcity,
    assembler/src/projects/mts, assembler/src/spades_pipeline
    xrstools # confirm, in WIZARD
    zlmdb # confirm, in _database
    ```

    I don't know if all these codepaths are actually ever reached, or tests
    ever run, so the packages won't necessarily break (or the breakage is
    very minor, such as a single community plugin in ansible). I don't
    guarantee there are no omissions from the list though.

    There is a significantly longer list of packages which appear to contain
    a use of yaml.load _somewhere_, but it is usually in some
    maintainer/release script, or an optional path somewhere, and the
    package itself doesn't [build-]depend on python3-yaml.




    Cheers
    Timo


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Louis-Philippe_V=c3=a9ron@21:1/5 to All on Mon Oct 10 17:50:01 2022
    This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------LiV0uqoN4tT10O5qdbzq0zO4
    Content-Type: multipart/mixed; boundary="------------fPZlPU67YOoaPt7DQ0vcDBNN"

    --------------fPZlPU67YOoaPt7DQ0vcDBNN
    Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

    T24gMjAyMi0xMC0wNiAyMSBoIDQzLCBQYXVsIFdpc2Ugd3JvdGU6DQo+IE9uIEZyaSwgMjAy Mi0xMC0wNyBhdCAwMDoxMCArMDIwMCwgR29yZG9uIEJhbGwgd3JvdGU6DQo+IA0KPj4gKiBV cGxvYWQgdG8gdW5zdGFibGUgYW5kIHNlZSB3aGF0IGJyZWFrcz8NCj4gDQo+IFRoZSBleHBl cmltZW50YWwgcHNldWRvLWV4Y3VzZXMgYWxyZWFkeSBzYXkgc2V2ZXJhbCBwYWNrYWdlcyBi cmVhazoNCj4gDQo+ICAgICBodHRwczovL3FhLmRlYmlhbi5vcmcvZXhjdXNlcy5waHA/ZXhw ZXJpbWVudGFsPTEmcGFja2FnZT1weXlhbWwNCj4gICAgIA0KPiAgICAgYXV0b3BrZ3Rlc3Qg Zm9yIGdhbmV0aS8zLjAuMi0xOiBhbWQ2NDogUmVncmVzc2lvbiwgYXJtNjQ6IFJlZ3Jlc3Np b24NCj4gICAgIGF1dG9wa2d0ZXN0IGZvciBsbHZtLXRvb2xjaGFpbi0xMy8xOjEzLjAuMS03 OiBhbWQ2NDogUGFzcywgYXJtNjQ6IFJlZ3Jlc3Npb24NCj4gICAgIGF1dG9wa2d0ZXN0IGZv ciBzYXRweS8wLjM3LjEtMTogYW1kNjQ6IFJlZ3Jlc3Npb24sIGFybTY0OiBSZWdyZXNzaW9u DQo+ICAgICBhdXRvcGtndGVzdCBmb3Igc3BhZGVzLzMuMTUuNStkZnNnLTE6IGFtZDY0OiBS ZWdyZXNzaW9uDQo+IA0KPiBTbyBhdCBsZWFzdCB0aGVzZSBpc3N1ZXMgbmVlZCB0byBiZSBp bnZlc3RpZ2F0ZWQgYW5kIG1heWJlIGJ1Z3MgZmlsZWQuDQo+IA0KPj4gKiBSZXF1ZXN0IGFu IGFyY2hpdmUgcmVidWlsZCB3aXRoIHRoaXMgdmVyc2lvbiBhbmQgc2VlIHdoYXQgYnJlYWtz Pw0KPiANCj4gRGVmaW5pdGVseS4NCj4gDQo+PiAqIEZpbGUgYnVncyBhZ2FpbnN0IGFsbCBs aWtlbHkgYWZmZWN0ZWQgcGFja2FnZXMgd2l0aCBhIGZpeGVkIGRhdGUgZm9yDQo+PiBhbiB1 cGxvYWQ/DQo+IA0KPiBEZWZpbml0ZWx5Lg0KDQpTaGFtZWxlc3MgcGx1ZyBoZXJlLCBidXQg aWYgeW91IGRvIGl0IGluIHRpbWUgZm9yIHRoZSBEUFQgc3ByaW50IA0KKERlY2VtYmVyIDIt My00KSwgZml4aW5nIHBhY2thZ2VzIHRoYXQgYnJva2UgY2FuIGV2ZW4gYmUgYW4gYWdlbmRh IGl0ZW0gDQpwZW9wbGUgY2FuIHdvcmsgb24hDQoNCmh0dHBzOi8vZGViLmxpLzJTOFUNCg0K LS0gDQogICDiooDio7TioL7ioLviorbio6bioIANCiAgIOKjvuKggeKioOKgkuKggOKjv+Kh gSAgTG91aXMtUGhpbGlwcGUgVsOpcm9ubmVhdQ0KICAg4qK/4qGE4qCY4qC34qCa4qCLICAg cG9sbG9AZGViaWFuLm9yZyAvIHZlcm9ubmVhdS5vcmcNCiAgIOKgiOKgs+KjhA0KDQo= --------------fPZlPU67YOoaPt7DQ0vcDBNN
    Content-Type: application/pgp-keys; name="OpenPGP_0xE1E5457C8BAD4113.asc" Content-Disposition: attachment; filename="OpenPGP_0xE1E5457C8BAD4113.asc" Content-Description: OpenPGP public key
    Content-Transfer-Encoding: quoted-printable

    -----BEGIN PGP PUBLIC KEY BLOCK-----

    xjMEYEPdjBYJKwYBBAHaRw8BAQdA5yh8SOHhcvKeX/A4rv0/JTCL8Kgnnwy4/okK h1Htbs3NOExvdWlzLVBoaWxpcHBlIFbDqXJvbm5lYXUgPGxvdWlzLXBoaWxpcHBl QHZlcm9ubmVhdS5vcmc+wpkEExYKAEECGwMFCQHhM4AFCwkIBwMFFQoJCAsFFgID AQACHgECF4AWIQT2TWHTIfPLSJFWdT3h5UV8i61BEwUCYEPeHgIZAQAKCRDh5UV8 i61BE0xKAP4oRsMaA2T/Zjge126dwHbnxBsjI/Q3ky8QkGlOffUKJAEA9dWm0hE4 0URSXM8Ndtf+GeHxvNeryVMCtVDUfjHMBA/CmQQTFgoAQQIbAwULCQgHAwUVCgkI CwUWAgMBAAIeAQIXgAIZARYhBPZNYdMh88tIkVZ1PeHlRXyLrUETBQJiEWgLBQkD rr3/AAoJEOHlRXyLrUETOK0BAM9I6BMMiqhsORsRcDVcM4VTm8G67YHapBW5zdl/ llfxAPwLAsi32TCPWjuwD3UdKig+6syvKFsiIfjiNBweNIQED80sTG91aXMtUGhp bGlwcGUgVsOpcm9ubmVhdSA8cG9sbG9AZGViaWFuLm9yZz7ClgQTFgoAPhYhBPZN YdMh88tIkVZ1PeHlRXyLrUETBQJgQ93rAhsDBQkB4TOABQsJCAcDBRUKCQgLBRYC AwEAAh4BAheAAAoJEOHlRXyLrUETeLMBAJAAznKkFo3Cm0pAW6klHv6jnDeMLS/6 9tAbJQRDNEAhAQDGQTrcAJZAcAFKoYeh2UlRokm1xG3Lc+FDpZGOKJBaBcKWBBMW CgA+AhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAFiEE9k1h0yHzy0iRVnU94eVF fIutQRMFAmIRaAsFCQOuvf8ACgkQ4eVFfIutQRMItwD+Oce5l0QBRJsax1C5MXe3 7Jk5cIMV2eOH0i4hd6c2wqYA/31Wn0qt5bv7i1y+2JsCeKtv0MIsYQ3LU1XG8k9h pb8BzjMEYEPg0RYJKwYBBAHaRw8BAQdASbekNA3xJnxUhMenK8ttfm8OTepniXHJ EN0Sm1/zmifCwDUEGBYKACYWIQT2TWHTIfPLSJFWdT3h5UV8i61BEwUCYEPg0QIb AgUJAeEzgACBCRDh5UV8i61BE3YgBBkWCgAdFiEEyqdABweoFrAgL8PN9CV6ULIc +oUFAmBD4NEACgkQ9CV6ULIc+oWswwEAoRTzlukc6Ss4PaChogmudTzMdezF1FQz T5HH0C4EVawA/1JfaysK+seL/zdEQKUHD3cMdg8NvMtOXfcMg4EiFRYE1SQBAPKi UCqSMLql7QtWiB/xmDFUYltNa3+NLjRYRsNKfe9JAP9ZEaXY6oO+3owwpxbNphBp hSkH+9lEag0Dd3BEowOKDMLANQQYFgoAJgIbAhYhBPZNYdMh88tIkVZ1PeHlRXyL rUETBQJiEnvDBQkDr85yAIF2IAQZFgoAHRYhBMqnQAcHqBawIC/DzfQlelCyHPqF BQJgQ+DRAAoJEPQlelCyHPqFrMMBAKEU85bpHOkrOD2goaIJrnU8zHXsxdRUM0+R x9AuBFWsAP9SX2srCvrHi/83REClBw93DHYPDbzLTl33DIOBIhUWBAkQ4eVFfIut QRPY6AEAn9YvrTzliAvnyPef3kXXCvyH973dPn/539suXireBnsA/iqtwiOe4758 +28fgsXaVUpyFcEhirsu0/IhzSnpVXUNzjgEYEPg5RIKKwYBBAGXVQEFAQEHQIES 2w30v+hi13deaiPcx7KPVMCUIA25nu6by9Wfa5BuAwEIB8J+BBgWCgAmFiEE9k1h 0yHzy0iRVnU94eVFfIutQRMFAmBD4OUCGwwFCQHhM4AACgkQ4eVFfIutQRMNhgD9 HkVqB+Vy+F9EAzjHilHnSPft2xfLdhTrqzh6O0jEhqsA/2dd/AMSsZNAH8FYQKq3 Th+Hikj+jXXs+P9HYlULp1UHwn4EGBYKACYCGwwWIQT2TWHTIfPLSJFWdT3h5UV8 i61BEwUCYhJ72AUJA6/OcwAKCRDh5UV8i61BE2CVAP9+JHidrPFWE7WwNskxdVY1 YzHxGihO20Zt65AagSMVgAD9FlBCTPfQKpvC5jBax89pLAg07QsLq1wJ5U5v1zV5 JQTOMwRiEWorFgkrBgEEAdpHDwEBB0BkhUACsGCOaaPRY4H2lJiegjp8hFrduGkl t4qxMygJ88J4BCgWCgAgFiEE9k1h0yHzy0iRVnU94eVFfIutQRMFAmLoLeYCHQMA CgkQ4eVFfIutQROVZAD9E2NDG9xBqa7gZjYprQkY4EzUgUkZY5g5l046jI0WvN8B APK0Ab4Sjx7ekPJDDa4gB/Mr1htCyoZrPysKB7tkuCQDwsA1BBgWCgAmFiEE9k1h 0yHzy0iRVnU94eVFfIutQRMFAmIRaisCGwIFCQHhM4AAgQkQ4eVFfIutQRN2IAQZ FgoAHRYhBJBd8+ORq1094UcSk2a2zWq+wNuWBQJiEWorAAoJEGa2zWq+wNuWOv8B AKfeLq2soJeiHDAdoV0spQxoVJDme2FzgmBCxr0KxRfQAP9zaHwI9+NjirmC8Gov IGveZ7wxXJ/v8jYFnZadVhIRBqk+AQDXKlTmPsWLD6SnMvW+kF1SbHUq6aPqALXb nEai/hTTrAD+Pt7NZO1KqJQiIJ+miP1LIlPqiZKMPt8uNdw8KKqHVwbOOARiEXES EgorBgEEAZdVAQUBAQdAZSMCxsNHkDiI2tnp9FX1Xl+39/Knre9jd7exta0LGAED AQgHwngEKBYKACAWIQT2TWHTIfPLSJFWdT3h5UV8i61BEwUCYuguFwIdAwAKCRDh 5UV8i61BE3D3APsH9gDArOrY6/d2/Lefpymj+yR5DHDEWpEvQ+GTnnA9ewEA6LgH Gx3DRN/KfkW1eoXxlnaFeQPXqggLOFj8kzYkDgDCfQQYFgoAJhYhBPZNYdMh88tI kVZ1PeHlRXyLrUETBQJiEXESAhsMBQkB4TOAAAoJEOHlRXyLrUETinYA93idFyhp u054EVRbFz/ybVAlpGqkdt69+LYt3Cr0RIkBANARMMYd47lV/1/C1fWsemRuZDCd +BzH/o7byibkUa4O
    =hixQ
    -----END PGP PUBLIC KEY BLOCK-----

    --------------fPZlPU67YOoaPt7DQ0vcDBNN--

    --------------LiV0uqoN4tT10O5qdbzq0zO4--

    -----BEGIN PGP SIGNATURE-----

    iHUEARYKAB0WIQTKp0AHB6gWsCAvw830JXpQshz6hQUCY0Q+PgAKCRD0JXpQshz6 hcfjAQDSLNlLP4C2/KTDAWkw5FLUpb5n549mzAtzyf3FPUhQ6AEA/ngfQyRWji97 lu9/bH6fHR0I6LrFWmjrUFKI/fUZmgc=
    =6XmO
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gordon Ball@21:1/5 to Gordon Ball on Wed Oct 19 22:40:01 2022
    On 09/10/2022 21:39, Gordon Ball wrote:
    On 07/10/2022 01:13, Timo Röhling wrote:
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for
    an upload?
    * Wait until after the freeze?
    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks
    is probably in dire need of maintenance anyway.

    Yes, that's a good point.

    I've done some more investigation of reverse-depends and looking for
    likely bad patterns with codesearch.d.n, and the set of packages which
    A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is pyyaml and B) actually depend or build-depend on `python3-yaml` is
    smaller than I feared. This is what I came up with:

    ```
    ansible # confirm - in ansible_collections/community/general/plugins
    buildbot # confirm - in porttostable
    ceph # confirm, in civetweb/third_party/duktape, src/test/crimson docker-compose # confirm, in tests
    dose3 # confirm, in scripts
    elasticsearch-curator # confirm, in test/unit
    etm # confirm, in etmTk/data
    ganeti # confirm, in qa, test/py
    gnocchi # confirm, in gnocchi/gendoc
    gr-dab # confirm, in python/app
    jeepyb # confirm, in cmd/notify_impact
    knitpy # confirm, in knitpy
    labgrid # confirm, in remote/coordinator
    lecm # confirm, in lecm/configuration
    lirc # confirm, in lirc/database
    lqa # confirm, in lqa_api/job
    open-adventure # confirm, in tests
    owslib # confirm, in ogcapi
    policyd-rate-limit # confirm, in config, tests
    python-aptly # confirm, in publisher
    python-canmatrix # confirm, in canmatrix/formats/yaml
    python-multipart # confirm, in multipart/tests
    python-pybedtools # confirm, in test, contrib
    python-seqcluster # confirm, in install
    python-tempestconf # confirm, in config_tempest/profile
    qcat # confirm, in adapters
    qcengine # confirm, in config
    refstack-client # confirm, in refstack_client
    relatorio # confirm, in templates/chart
    ripe-atlas-tools # confirm, in tools/settings
    ros-genpy # confirm, in test
    ros-rosinstall # confirm, in test, src/rosinstall
    ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
    spades # confirm, in assembler/src/test/teamcity,
    assembler/src/projects/mts, assembler/src/spades_pipeline
    xrstools # confirm, in WIZARD
    zlmdb # confirm, in _database
    ```

    I don't know if all these codepaths are actually ever reached, or tests
    ever run, so the packages won't necessarily break (or the breakage is
    very minor, such as a single community plugin in ansible). I don't
    guarantee there are no omissions from the list though.

    There is a significantly longer list of packages which appear to contain
    a use of yaml.load _somewhere_, but it is usually in some
    maintainer/release script, or an optional path somewhere, and the
    package itself doesn't [build-]depend on python3-yaml.


    I've filed bugs for (most) of the packages listed above (some were
    dropped on review because they weren't in main, or the affected file
    appeared to be unused at build time and not installed). I've had a
    couple of acknowledgements/fixes already.

    A number of affected package look very sparsely used and/or maintained,
    and I doubt are likely to be fixed. However, of the packages above, only
    two have autopkgtests which fail on experimental migration, so I
    wouldn't be surprised if some are not already broken by other past changes.

    I propose to wait ~2 weeks until the beginning of November and upload
    pyyaml 6 to unstable then.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gordon Ball@21:1/5 to Gordon Ball on Wed Nov 2 09:50:01 2022
    On 19/10/2022 22:30, Gordon Ball wrote:
    On 09/10/2022 21:39, Gordon Ball wrote:
    On 07/10/2022 01:13, Timo Röhling wrote:
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for >>>> an upload?
    * Wait until after the freeze?
    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks
    is probably in dire need of maintenance anyway.

    Yes, that's a good point.

    I've done some more investigation of reverse-depends and looking for
    likely bad patterns with codesearch.d.n, and the set of packages which
    A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is
    pyyaml and B) actually depend or build-depend on `python3-yaml` is
    smaller than I feared. This is what I came up with:

    ```
    ansible # confirm - in ansible_collections/community/general/plugins
    buildbot # confirm - in porttostable
    ceph # confirm, in civetweb/third_party/duktape, src/test/crimson
    docker-compose # confirm, in tests
    dose3 # confirm, in scripts
    elasticsearch-curator # confirm, in test/unit
    etm # confirm, in etmTk/data
    ganeti # confirm, in qa, test/py
    gnocchi # confirm, in gnocchi/gendoc
    gr-dab # confirm, in python/app
    jeepyb # confirm, in cmd/notify_impact
    knitpy # confirm, in knitpy
    labgrid # confirm, in remote/coordinator
    lecm # confirm, in lecm/configuration
    lirc # confirm, in lirc/database
    lqa # confirm, in lqa_api/job
    open-adventure # confirm, in tests
    owslib # confirm, in ogcapi
    policyd-rate-limit # confirm, in config, tests
    python-aptly # confirm, in publisher
    python-canmatrix # confirm, in canmatrix/formats/yaml
    python-multipart # confirm, in multipart/tests
    python-pybedtools # confirm, in test, contrib
    python-seqcluster # confirm, in install
    python-tempestconf # confirm, in config_tempest/profile
    qcat # confirm, in adapters
    qcengine # confirm, in config
    refstack-client # confirm, in refstack_client
    relatorio # confirm, in templates/chart
    ripe-atlas-tools # confirm, in tools/settings
    ros-genpy # confirm, in test
    ros-rosinstall # confirm, in test, src/rosinstall
    ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
    spades # confirm, in assembler/src/test/teamcity,
    assembler/src/projects/mts, assembler/src/spades_pipeline
    xrstools # confirm, in WIZARD
    zlmdb # confirm, in _database
    ```

    I don't know if all these codepaths are actually ever reached, or tests
    ever run, so the packages won't necessarily break (or the breakage is
    very minor, such as a single community plugin in ansible). I don't
    guarantee there are no omissions from the list though.

    There is a significantly longer list of packages which appear to contain
    a use of yaml.load _somewhere_, but it is usually in some
    maintainer/release script, or an optional path somewhere, and the
    package itself doesn't [build-]depend on python3-yaml.


    I've filed bugs for (most) of the packages listed above (some were
    dropped on review because they weren't in main, or the affected file
    appeared to be unused at build time and not installed). I've had a
    couple of acknowledgements/fixes already.

    A number of affected package look very sparsely used and/or maintained,
    and I doubt are likely to be fixed. However, of the packages above, only
    two have autopkgtests which fail on experimental migration, so I
    wouldn't be surprised if some are not already broken by other past changes.

    I propose to wait ~2 weeks until the beginning of November and upload
    pyyaml 6 to unstable then.


    pyyaml 6 has now landed in unstable. About half the bugs I filed have
    been resolved, and there is only one package (ganeti) with autopkgtests blocking migration.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Kitterman@21:1/5 to Gordon Ball on Wed Nov 2 10:30:01 2022
    On November 2, 2022 8:35:35 AM UTC, Gordon Ball <gordon@chronitis.net> wrote: >On 19/10/2022 22:30, Gordon Ball wrote:
    On 09/10/2022 21:39, Gordon Ball wrote:
    On 07/10/2022 01:13, Timo Röhling wrote:
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks?
    * File bugs against all likely affected packages with a fixed date for >>>>> an upload?
    * Wait until after the freeze?
    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks
    is probably in dire need of maintenance anyway.

    Yes, that's a good point.

    I've done some more investigation of reverse-depends and looking for
    likely bad patterns with codesearch.d.n, and the set of packages which
    A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is >>> pyyaml and B) actually depend or build-depend on `python3-yaml` is
    smaller than I feared. This is what I came up with:

    ```
    ansible # confirm - in ansible_collections/community/general/plugins
    buildbot # confirm - in porttostable
    ceph # confirm, in civetweb/third_party/duktape, src/test/crimson
    docker-compose # confirm, in tests
    dose3 # confirm, in scripts
    elasticsearch-curator # confirm, in test/unit
    etm # confirm, in etmTk/data
    ganeti # confirm, in qa, test/py
    gnocchi # confirm, in gnocchi/gendoc
    gr-dab # confirm, in python/app
    jeepyb # confirm, in cmd/notify_impact
    knitpy # confirm, in knitpy
    labgrid # confirm, in remote/coordinator
    lecm # confirm, in lecm/configuration
    lirc # confirm, in lirc/database
    lqa # confirm, in lqa_api/job
    open-adventure # confirm, in tests
    owslib # confirm, in ogcapi
    policyd-rate-limit # confirm, in config, tests
    python-aptly # confirm, in publisher
    python-canmatrix # confirm, in canmatrix/formats/yaml
    python-multipart # confirm, in multipart/tests
    python-pybedtools # confirm, in test, contrib
    python-seqcluster # confirm, in install
    python-tempestconf # confirm, in config_tempest/profile
    qcat # confirm, in adapters
    qcengine # confirm, in config
    refstack-client # confirm, in refstack_client
    relatorio # confirm, in templates/chart
    ripe-atlas-tools # confirm, in tools/settings
    ros-genpy # confirm, in test
    ros-rosinstall # confirm, in test, src/rosinstall
    ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
    spades # confirm, in assembler/src/test/teamcity,
    assembler/src/projects/mts, assembler/src/spades_pipeline
    xrstools # confirm, in WIZARD
    zlmdb # confirm, in _database
    ```

    I don't know if all these codepaths are actually ever reached, or tests
    ever run, so the packages won't necessarily break (or the breakage is
    very minor, such as a single community plugin in ansible). I don't
    guarantee there are no omissions from the list though.

    There is a significantly longer list of packages which appear to contain >>> a use of yaml.load _somewhere_, but it is usually in some
    maintainer/release script, or an optional path somewhere, and the
    package itself doesn't [build-]depend on python3-yaml.


    I've filed bugs for (most) of the packages listed above (some were dropped on review because they weren't in main, or the affected file appeared to be unused at build time and not installed). I've had a couple of acknowledgements/fixes already.

    A number of affected package look very sparsely used and/or maintained, and I doubt are likely to be fixed. However, of the packages above, only two have autopkgtests which fail on experimental migration, so I wouldn't be surprised if some are not
    already broken by other past changes.

    I propose to wait ~2 weeks until the beginning of November and upload pyyaml 6 to unstable then.


    pyyaml 6 has now landed in unstable. About half the bugs I filed have been resolved, and there is only one package (ganeti) with autopkgtests blocking migration.


    There were two more, but I retried the migration reference jobs and they failed (which makes them no longer block since they are no longer a regression).

    Ganeti can't currently be fixed since it's unbuildable. It's only due to an archive hiccup that it's even in Testing. I was planning on asking the Release Team to ignore it, so pyyaml can transition (worst case it's stuck until ganeti gets removed from
    Testing in a week and a half).

    Scott K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Kitterman@21:1/5 to All on Sat Nov 12 21:29:24 2022
    On Wednesday, November 2, 2022 5:26:35 AM EST Scott Kitterman wrote:
    On November 2, 2022 8:35:35 AM UTC, Gordon Ball <gordon@chronitis.net>
    wrote:
    On 19/10/2022 22:30, Gordon Ball wrote:
    On 09/10/2022 21:39, Gordon Ball wrote:
    On 07/10/2022 01:13, Timo Rhling wrote:
    Hi Gordon,

    * Gordon Ball <gordon@chronitis.net> [2022-10-07 00:10]:
    * Upload to unstable and see what breaks?
    * Request an archive rebuild with this version and see what breaks? >>>>> * File bugs against all likely affected packages with a fixed date for >>>>> an upload?
    * Wait until after the freeze?

    Considering that PyYAML has been issuing a YAMLLoadWarning since
    version 5.1 (i.e. September 2019), I would expect that many (most?)
    reverse dependencies have been fixed, and anything that still breaks >>>> is probably in dire need of maintenance anyway.

    Yes, that's a good point.

    I've done some more investigation of reverse-depends and looking for
    likely bad patterns with codesearch.d.n, and the set of packages which >>> A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is >>> pyyaml and B) actually depend or build-depend on `python3-yaml` is
    smaller than I feared. This is what I came up with:

    ```
    ansible # confirm - in ansible_collections/community/general/plugins
    buildbot # confirm - in porttostable
    ceph # confirm, in civetweb/third_party/duktape, src/test/crimson
    docker-compose # confirm, in tests
    dose3 # confirm, in scripts
    elasticsearch-curator # confirm, in test/unit
    etm # confirm, in etmTk/data
    ganeti # confirm, in qa, test/py
    gnocchi # confirm, in gnocchi/gendoc
    gr-dab # confirm, in python/app
    jeepyb # confirm, in cmd/notify_impact
    knitpy # confirm, in knitpy
    labgrid # confirm, in remote/coordinator
    lecm # confirm, in lecm/configuration
    lirc # confirm, in lirc/database
    lqa # confirm, in lqa_api/job
    open-adventure # confirm, in tests
    owslib # confirm, in ogcapi
    policyd-rate-limit # confirm, in config, tests
    python-aptly # confirm, in publisher
    python-canmatrix # confirm, in canmatrix/formats/yaml
    python-multipart # confirm, in multipart/tests
    python-pybedtools # confirm, in test, contrib
    python-seqcluster # confirm, in install
    python-tempestconf # confirm, in config_tempest/profile
    qcat # confirm, in adapters
    qcengine # confirm, in config
    refstack-client # confirm, in refstack_client
    relatorio # confirm, in templates/chart
    ripe-atlas-tools # confirm, in tools/settings
    ros-genpy # confirm, in test
    ros-rosinstall # confirm, in test, src/rosinstall
    ros-rosinstall-generator # confirm, in test, src/rosinstall_generator
    spades # confirm, in assembler/src/test/teamcity,
    assembler/src/projects/mts, assembler/src/spades_pipeline
    xrstools # confirm, in WIZARD
    zlmdb # confirm, in _database
    ```

    I don't know if all these codepaths are actually ever reached, or tests >>> ever run, so the packages won't necessarily break (or the breakage is
    very minor, such as a single community plugin in ansible). I don't
    guarantee there are no omissions from the list though.

    There is a significantly longer list of packages which appear to contain >>> a use of yaml.load _somewhere_, but it is usually in some
    maintainer/release script, or an optional path somewhere, and the
    package itself doesn't [build-]depend on python3-yaml.

    I've filed bugs for (most) of the packages listed above (some were
    dropped on review because they weren't in main, or the affected file
    appeared to be unused at build time and not installed). I've had a
    couple of acknowledgements/fixes already.

    A number of affected package look very sparsely used and/or maintained,
    and I doubt are likely to be fixed. However, of the packages above, only >> two have autopkgtests which fail on experimental migration, so I
    wouldn't be surprised if some are not already broken by other past
    changes.

    I propose to wait ~2 weeks until the beginning of November and upload
    pyyaml 6 to unstable then.>
    pyyaml 6 has now landed in unstable. About half the bugs I filed have been >resolved, and there is only one package (ganeti) with autopkgtests
    blocking migration.
    There were two more, but I retried the migration reference jobs and they failed (which makes them no longer block since they are no longer a regression).

    Ganeti can't currently be fixed since it's unbuildable. It's only due to an archive hiccup that it's even in Testing. I was planning on asking the Release Team to ignore it, so pyyaml can transition (worst case it's stuck until ganeti gets removed from Testing in a week and a half).

    Ganeti's removal got delays, but the Release Team hinted pyyaml to transition, so it should go to Testing now.

    Scott K
    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmNwVoQACgkQeNfe+5rV mvG2Ig//cYxDyMl746MlhSnF6rjF6wOy8E2+mbImbraMrOiS/eS9BUEpMHT24885 vWpnV0KWxa9/WW50TDKTRN1N54Mfp5kfW6K/Kr0sheViloV/t8M2foDxP4nFrL6L MzqelKrVVSfeZGPsbNKAHDDpdEYmLgP1mgdPq4ncKReJZxVTM9u52j4YHsXfizJ1 59cwGnOenb1X5A9keTPWjHn07ueDPbIeoJ6YtK5oy2hM31hY0kiNjuJzk4bYJCTe qD8Zzb31HAuRhV+g2ZyyvITZVesgbmSVSqlFD8g2k2KFjEyXfyMFN8ttsLXT9xZe KcDML08DVsjQNvx4sPiWztGMUksJ94hm4mTuG/BMum6ixOzrOEi+GZAAHblOinrq /ZW02rcrUdC6TA/05+HguHysyFb6vDnOEvW9Ng0/sHhBXNDi94SYGx1XsyNcqp0v 5WuAO/eMyLV6KPX8ji4S+pnNWwIoIqNYQ4qp0vCTTp39Ge+tzA1utm8Ht7IedgFH JJIEIep/CgDxnKi7HaoglQOQl+Y4xPeTmbdMrI0SL3OeQPFWf2EVMxRth/O0hwpB E7e1fD1KGzLR9j3p+RP8QPETTLC5cHJmgiOoAvBv5Gfu4mhpIeJvOZMo1HwLvtBG qTOsHKMDaWtutnVIK9wJMQQAByW9iY2i6XdI2rSeECsCd7Sz53w=
    =IvK7
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)