1. Forum
  2. Usenet
  3. LINUX.DEBIAN.MAINT.PYTHON

  • Different behavior of package with pyproject.toml

    From Ole Streicher@21:1/5 to All on Sat Dec 3 17:50:01 2022
    Hi,

    I need some help with a package that switched to using pyproject.toml
    only. The package is asdf-astropy, and the problem I have that it does
    not package all Python files recursively: for example astropy_asdf.io
    files are missing (as seen in unstable now).

    https://tracker.debian.org/pkg/asdf-astropy https://salsa.debian.org/debian-astro-team/asdf-astropy

    What I do not understand is that this happens "sometimes":

    * when I build the package in a clean schroot with "debuild", files in
    subpackages are missing (but no error message in the log)

    * when I install git, then a debuild results in a complete package

    * when I cut&paste the command from the build log [1], the copy process
    always happens completely, independent of whether git was installed or
    not

    * when I use pbuilder, files of subpackages are never package, with our
    without git in the build dependencies

    [1] python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir .pybuild/cpython3_3.11

    Since the package "sometimes" is built correctly, I think that upstream
    did it right; however I have no idea where the problem could be. Does
    anyone have an idea?

    Best

    Ole

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Kitterman@21:1/5 to All on Sat Dec 3 20:56:53 2022
    On Saturday, December 3, 2022 11:19:06 AM EST Ole Streicher wrote:
    Hi,

    I need some help with a package that switched to using pyproject.toml
    only. The package is asdf-astropy, and the problem I have that it does
    not package all Python files recursively: for example astropy_asdf.io
    files are missing (as seen in unstable now).

    https://tracker.debian.org/pkg/asdf-astropy https://salsa.debian.org/debian-astro-team/asdf-astropy

    What I do not understand is that this happens "sometimes":

    * when I build the package in a clean schroot with "debuild", files in
    subpackages are missing (but no error message in the log)

    * when I install git, then a debuild results in a complete package

    * when I cut&paste the command from the build log [1], the copy process
    always happens completely, independent of whether git was installed or
    not

    * when I use pbuilder, files of subpackages are never package, with our
    without git in the build dependencies

    [1] python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir .pybuild/cpython3_3.11

    Since the package "sometimes" is built correctly, I think that upstream
    did it right; however I have no idea where the problem could be. Does
    anyone have an idea?

    My first guess is that in some circumstances setuptools is doing the installing and in others it's the dh-python pyproject plugin (using the installer
    module). Looking at the build log on buildd.d.o, I can see that build is
    using the plugin/installer.

    I think it would be useful to stop the build right before the install step starts [1], manually unpack the wheel (it's a zip file) and see if it has all the files in it. If it does, then plugin/build is doing the right thing and it's an issue with either the plugin or the installer module.

    Since the build stage uses setuptools as part of it's build process, I thing it's most likely an installer issue. I recently found the installer module doesn't correctly handle flit data files, but I was able to work around that with appropriate .install files.

    Scott K

    [1] pybuild plugin_pyproject:118: Unpacking wheel built for python3.10 with "installer" module
    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmOL/mUACgkQeNfe+5rV mvFMfxAA1dgzpX+RrTkWOfO4KTueFHd1DHHKzQjEMyGYoyqOUFgPuA6CrdDJ9pqI nG6p7YfOxP5kGhmHJ1le8pZgEW2Ezy0UFau2ReJ2OHoC5Nv1Mtm8qYIDFUyZhziK syIIVxTIu9IAndXq4EY4bpGHGd1jK7FsTEiKEGGxuW+19fkgSYvxFonNoLHzh2nn B/nQOf2Yy6I8suu8TyQbjI7682ZYi+m/aA9F3zLB+a7hiYn9h6Qe2xgoodhahj0T XMrzA+YWyuQTvO+3dXeDsvvNN/ium104YoPpP7uvejN7biRU/XqbWFfGxzPqZNux uDX5b1raQa2xN7ASlJBivolTSaiQtsN1TB95of4lm0q43mg+ihb5QEXKGrv4W/0N ITIDTnOscPuLRlKeXka9VWwLorqNJ6cRBkzm7mQuFi8Lzoj2vXIZQi4eyXux/8RR n8ZXD39qScCFSrY4wghvZZAsj2XeQ8qhkiQyOP6uJHDapkCMYk4cAoIC7itLBYkO euuonmRbsuBdJF17X2+Vv1jiX9RD+bxkKIg9RrXtPnl2tDnnmI3b76L5GRR8qe2W OAqVPI0j5TfFVuDp0LtGvcSQvpyoaDV0H/neaUaMh56zJDiYhyPM3my+2hybZ47z OEbPRdv8GjK37jViR9GOzg+jJ0l44m8gOsamO0fsf3CG5ucqlVk=
    =N2oY
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ole Streicher@21:1/5 to Scott Kitterman on Sun Dec 4 17:10:01 2022
    Hi Scott,

    thanks for the hint.

    Scott Kitterman <debian@kitterman.com> writes:
    My first guess is that in some circumstances setuptools is doing the installing
    and in others it's the dh-python pyproject plugin (using the installer module). Looking at the build log on buildd.d.o, I can see that build is using the plugin/installer.

    I think it would be useful to stop the build right before the install step starts [1], manually unpack the wheel (it's a zip file) and see if it has all the files in it. If it does, then plugin/build is doing the right thing and it's an issue with either the plugin or the installer module.

    I couldn't manage this; however when looking into the log, I see

    ---------------------------------8<---------------------------------------------
    dh_auto_build -O--buildsystem=pybuild
    I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" module
    I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir /builds/debian-astro-team/asdf-astropy/debian/output/source_dir/.pybuild/cpython3_3.11
    /usr/lib/python3/dist-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/asdf_astropy
    copying asdf_astropy/_version.py -> build/lib/asdf_astropy
    [... no asdf_astropy/io files here ...]
    * Building wheel...
    Successfully built asdf_astropy-0.3.0-py3-none-any.whl
    I: pybuild plugin_pyproject:118: Unpacking wheel built for python3.11 with "installer" module
    ---------------------------------8<---------------------------------------------

    So I would suspect that it is already the build does something wrong
    here. Which package is that? pybuild-plugin-pyproject?

    Best

    Ole

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ole Streicher@21:1/5 to Scott Kitterman on Sun Dec 4 18:10:01 2022
    Scott Kitterman <debian@kitterman.com> writes:
    On December 4, 2022 3:42:15 PM UTC, Ole Streicher <olebole@debian.org> wrote:
    So I would suspect that it is already the build does something wrong
    here. Which package is that? pybuild-plugin-pyproject?


    python3-installer


    Thank you, I filed a bug.

    Best

    Ole

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)