• Re: Helping hands needed to upgrade scipy

    From Jerome Kieffer@21:1/5 to Andreas Tille on Tue Jan 17 18:50:01 2023
    Hi Andreas,

    I am an upstream author of software packages (fabio & pyFAI) which took
    the same build system as scipy (meson-python) and I can confirm you
    there are issues with the debian packaging:

    * finding cython (apparently it should be fixed upstream in meson)
    * choose the proper python: sid has currently py3.10 and 3.11
    co-installed and I found out the wrong one was used when building: https://github.com/mesonbuild/meson/issues/11248

    I hope this helps,

    Jerome


    On Tue, 17 Jan 2023 18:12:42 +0100
    Andreas Tille <andreas@an3as.eu> wrote:

    Hi,

    I upgraded the experimental branch of scipy to contain the submodules upstream includes. I'm *not* convinced that we need them all -
    hopefully we can get rid of boost (but upstream has some patches here)
    and scipy-mathjax.

    For the moment I've tried to build the experimental branch and ended
    up with some issue where cython can't be found (should be cython3 for sure)[1].

    Unfortunately my spare time is occupied now by other more urgent tasks.
    I'd be happy if someone else could review / pick up / continue from
    here.

    Kind regards
    Andreas.


    [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903

    --
    http://fam-tille.de


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Tue Jan 17 18:20:01 2023
    Hi,

    I upgraded the experimental branch of scipy to contain the submodules
    upstream includes. I'm *not* convinced that we need them all -
    hopefully we can get rid of boost (but upstream has some patches here)
    and scipy-mathjax.

    For the moment I've tried to build the experimental branch and ended
    up with some issue where cython can't be found (should be cython3 for
    sure)[1].

    Unfortunately my spare time is occupied now by other more urgent tasks.
    I'd be happy if someone else could review / pick up / continue from
    here.

    Kind regards
    Andreas.


    [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Drew Parsons@21:1/5 to Andreas Tille on Wed Jan 18 02:50:01 2023
    On 2023-01-17 18:12, Andreas Tille wrote:
    Hi,

    I upgraded the experimental branch of scipy to contain the submodules upstream includes. I'm *not* convinced that we need them all -
    hopefully we can get rid of boost (but upstream has some patches here)
    and scipy-mathjax.

    For the moment I've tried to build the experimental branch and ended
    up with some issue where cython can't be found (should be cython3 for sure)[1].

    Unfortunately my spare time is occupied now by other more urgent tasks.
    I'd be happy if someone else could review / pick up / continue from
    here.


    The problem was not meson as such, but that upstream is only half using
    meson. They must have decided they don't trust the way meson handles
    cython, and set up their own custom build tools for it, see https://github.com/scipy/scipy/pull/15407

    That is, they hard-coded the cython executable name in scipy/_build_utils/cythoner.py

    They actually did the same already in the old setuptools build (tools/cythonize.py), but we didn't notice since they had the breaking
    cython call wrapped in a try block which inspected a Cython class from
    inside python. I raised an Issue about whether cythoner.py should do
    the same in https://github.com/scipy/scipy/issues/17808
    We could argue the fault is debian's for not providing a cython
    executable (it does have -2 and -3 options for choosing the python
    version to work with).

    I figure also we should use python3-mesonpy, since it's what upstream
    uses. Best not to diverge too far from their build method I think.

    I've pushed commits updating cythoner.py to point cythoner.py at
    cython3. The main build now completes. Have to clean up test handling
    next: "FileNotFoundError: [Errno 2] No such file or directory: '/projects/python/build/scipy/.coveragerc'

    Drew

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Wed Jan 18 08:30:02 2023
    Hi Drew,

    Am Wed, Jan 18, 2023 at 02:45:31AM +0100 schrieb Drew Parsons:
    On 2023-01-17 18:12, Andreas Tille wrote:
    Hi,

    I upgraded the experimental branch of scipy to contain the submodules upstream includes. I'm *not* convinced that we need them all -
    hopefully we can get rid of boost (but upstream has some patches here)
    and scipy-mathjax.

    For the moment I've tried to build the experimental branch and ended
    up with some issue where cython can't be found (should be cython3 for sure)[1].

    Unfortunately my spare time is occupied now by other more urgent tasks.
    I'd be happy if someone else could review / pick up / continue from
    here.


    The problem was not meson as such, but that upstream is only half using meson. They must have decided they don't trust the way meson handles
    cython, and set up their own custom build tools for it, see https://github.com/scipy/scipy/pull/15407

    That is, they hard-coded the cython executable name in scipy/_build_utils/cythoner.py

    They actually did the same already in the old setuptools build (tools/cythonize.py), but we didn't notice since they had the breaking
    cython call wrapped in a try block which inspected a Cython class from
    inside python. I raised an Issue about whether cythoner.py should do the same in https://github.com/scipy/scipy/issues/17808
    We could argue the fault is debian's for not providing a cython executable (it does have -2 and -3 options for choosing the python version to work with).

    Thanks for the clarification.

    I figure also we should use python3-mesonpy, since it's what upstream uses. Best not to diverge too far from their build method I think.

    I'm perfectly fine with this approach. That's why I initially pushed meson-python to new. I've just dput meson-python_0.12.0-2_source to let
    it migrate to testing. My attempt to use plain meson (as per upstream
    readme) was just to shorten the time while meson-python was in new
    queue.

    I've pushed commits updating cythoner.py to point cythoner.py at cython3.
    The main build now completes. Have to clean up test handling next: "FileNotFoundError: [Errno 2] No such file or directory: '/projects/python/build/scipy/.coveragerc'

    Nice. Hope my preparation (of meson-python and the multi-source tarball) helped to move forward. Please double check whether we really need the
    boost code copy.

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)