• Migration from setuptools to pyproject

    From Jerome Kieffer@21:1/5 to All on Wed May 3 10:20:01 2023
    Dear Debian Packager

    I am the upstream developer of a few debian packages (fabio, pyfai...) and those packages used to rely on numpy.distutils. Since setuptools 60,
    this is not more possible so I changed the build system to use mesonpy
    (pep517 compliant).

    I found it easy to tell the debian packaging helper pybuild to use the `pyproject.toml` file to build the software instead of the `setup.py`:
    `export PYBUILD_SYSTEM=pyproject` in the debian/rules file.

    But later on, the meson configuration finds cython as cython3 but complains the compiler was not found:

    ```
    dh build --with python3,sphinxdoc --buildsystem=pybuild
    dh_update_autotools_config -O--buildsystem=pybuild
    dh_autoreconf -O--buildsystem=pybuild
    dh_auto_configure -O--buildsystem=pybuild
    pybuild --configure -i python{version} -p 3.11
    dh_auto_build -O--buildsystem=pybuild
    pybuild --build -i python{version} -p 3.11
    I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" module
    I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.pybuild/cpython3_3.11_pyfai
    * Building wheel...
    + meson setup --prefix=/usr /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0 /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build --native-file=/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.
    mesonpy-native-file.ini -Ddebug=false -Doptimization=2
    The Meson build system
    Version: 1.0.1
    Source dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0
    Build dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
    Build type: native build
    Project name: pyFAI
    Project version: 2023.5.1a0
    C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
    C linker for the host machine: cc ld.bfd 2.40
    Cython compiler for the host machine: cython3 (cython 0.29.32)
    Host machine cpu family: x86_64
    Host machine cpu: x86_64
    Program cython found: NO

    ../../meson.build:17:0: ERROR: Program 'cython' not found or not executable

    ```

    I had a similar problem for python3 and it was solved by installing `python-is-python3`. But there is no such trick for cython ...

    Nota: `sudo ln -s /usr/bin/cython3 /usr/bin/cython` addresses the bug but I am looking for a solution which is compliant with debian packaging !

    Thanks for your help.

    Jerome

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jerome Kieffer@21:1/5 to pollo@debian.org on Thu May 4 08:40:01 2023
    Dear Louis-Philippe,

    Thankd for your answer ... I will try to give some precision, inlined in the text...

    On Wed, 3 May 2023 11:13:56 -0400
    Louis-Philippe VĂ©ronneau <pollo@debian.org> wrote:

    On 2023-05-03 04 h 06, Jerome Kieffer wrote:
    Dear Debian Packager

    I am the upstream developer of a few debian packages (fabio, pyfai...) and those packages used to rely on numpy.distutils. Since setuptools 60,
    this is not more possible so I changed the build system to use mesonpy (pep517 compliant).

    I found it easy to tell the debian packaging helper pybuild to use the `pyproject.toml` file to build the software instead of the `setup.py`:
    `export PYBUILD_SYSTEM=pyproject` in the debian/rules file.

    But later on, the meson configuration finds cython as cython3 but complains the compiler was not found:

    ```
    dh build --with python3,sphinxdoc --buildsystem=pybuild
    dh_update_autotools_config -O--buildsystem=pybuild
    dh_autoreconf -O--buildsystem=pybuild
    dh_auto_configure -O--buildsystem=pybuild
    pybuild --configure -i python{version} -p 3.11
    dh_auto_build -O--buildsystem=pybuild
    pybuild --build -i python{version} -p 3.11
    I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" module
    I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.pybuild/cpython3_3.11_pyfai
    * Building wheel...
    + meson setup --prefix=/usr /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0 /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build --native-file=/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.
    1a0/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
    The Meson build system
    Version: 1.0.1
    Source dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0 Build dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
    Build type: native build
    Project name: pyFAI
    Project version: 2023.5.1a0
    C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Debian 12.2.0-14) 12.2.0")
    C linker for the host machine: cc ld.bfd 2.40
    Cython compiler for the host machine: cython3 (cython 0.29.32)
    Host machine cpu family: x86_64
    Host machine cpu: x86_64
    Program cython found: NO

    ../../meson.build:17:0: ERROR: Program 'cython' not found or not executable

    ```

    I had a similar problem for python3 and it was solved by installing `python-is-python3`. But there is no such trick for cython ...

    Nota: `sudo ln -s /usr/bin/cython3 /usr/bin/cython` addresses the bug but I am looking for a solution which is compliant with debian packaging !

    Thanks for your help.

    Jerome


    Hi,

    It's hard to help you without seeing what the /debian dir looks like.

    Sorry, I forgot to put a link to the (ongoing) work: https://github.com/silx-kit/pyFAI/pull/1867/files

    This directory contains the debian files needed for packaging on debian 12/testing/unstable. Many things are still copy-paste from the former
    build system (numpy-distutils)

    In theory, you don't need to specify `PYBUILD_SYSTEM=pyproject` for
    pybuild to use the pyproject.toml file.

    Actually, there is still the former setup.py which is still used for
    packaging under debian11- and also conda on macos. All other packaging apparently support pyproject/meson-python.

    As per the pybuild manpage, you need to have the
    `pybuild-plugin-pyproject` package as a build-dependency and the
    package you use as the build system, in your case, `python3-mesonpy`.

    Thanks for reminding me.

    As for your cython problem, it doesn't seem like you are building in
    a clean environment. I would highly recommend you use something like
    sbuild to do so:

    https://wiki.debian.org/sbuild


    The computer was freshly installed with debian12 ... so no, it is not a
    build in a chroot/docker but it is fairly clean. My question was rather
    how to remove one level of abstraction to debug the code rather than
    how to add one to make it cleaner.

    Cheers,

    Jerome

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)