1. Forum
  2. Usenet
  3. LINUX.DEBIAN.MAINT.PYTHON

  • .egg-info for entry points during dh_auto_test

    From Michael Fladischer@21:1/5 to All on Thu Oct 21 21:30:01 2021
    Hi,

    I'm working on src:pytest-lazy-fixtures and was trying to get the
    unittests to run but it seems that I have run into a problem that I'm
    not sure on how to fix it in a clean way.

    pytest-lazy-fixtures is a pytest plugin and registers itself through the
    Python entrypoints mechanism. In its unitests, it assumes that this registration has already happened but during dh_auto_test there is no
    .egg-info directory available. I could use "python3 setup.py develop -x"
    to generate it, but this registers the package in /usr inside the build
    chroot which I think is not the best solution.

    Is there an other, less intrusive way to register the entrypoint before
    running the tests?

    Thanks,
    Michael

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefano Rivera@21:1/5 to All on Fri Oct 22 05:50:01 2021
    Hi Michael (2021.10.21_18:55:51_+0000)
    pytest-lazy-fixtures is a pytest plugin and registers itself through the Python entrypoints mechanism. In its unitests, it assumes that this registration has already happened but during dh_auto_test there is no .egg-info directory available. I could use "python3 setup.py develop -x" to generate it, but this registers the package in /usr inside the build chroot which I think is not the best solution.

    I looked at this recently for pytest-doctestplus.

    The lazy answer is to run the full test suite in an autopkgtest, not at
    build time.

    pytest also has a mechanism to register plugins through an environment variable, but I couldn't figure out how to get the test suite to work
    with that.

    If you do find a nice mechanism, we should probably apply it to the
    packaging of all pytest plugins.

    SR

    --
    Stefano Rivera
    http://tumbleweed.org.za/
    +1 415 683 3272

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stuart Prescott@21:1/5 to All on Fri Oct 22 09:20:01 2021
    Hi Michael

    I wrestled with something similar some years ago with translate-toolkit, which has a set of tests that run the binaries it ships but these (entry-points) binaries are not generated until after installation.

    The pybuild/debhelper sequence is to build, test then install; upstream's view is to build, install then test, and so that is what I faked for that package, overriding the normal test invocation, and running the tests in an execute_after_dh_install step:

    https://sources.debian.org/src/translate-toolkit/unstable/debian/rules/#L59

    I help the test suite find the binaries in debian/translate-toolkit/usr/bin and pybuild points the test harness at the modules in the normal place in .pybuild/.

    That's pretty ugly, but it has worked out ok for this package.

    A related problem is the PEP517 builders go from source to wheel and then install the wheel; the entry_points and scripts only appear after
    installation, and so once again, testing most probably lives after 'install' not before.

    Given that entry_points, plugin registration, PEP517 are all pointing us towards build/install/test as the paradigm rather than build/test/install, do we need to look for an additional dh step to be added to our normal sequence
    to help us with what is becoming a common task?

    regards
    Stuart

    --
    Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net Debian Developer http://www.debian.org/ stuart@debian.org
    GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael Fladischer@21:1/5 to All on Fri Oct 22 16:30:02 2021
    Hi Stefano,

    Am 22.10.2021 um 05:44 schrieb Stefano Rivera:
    pytest also has a mechanism to register plugins through an environment variable, but I couldn't figure out how to get the test suite to work
    with that.

    thanks for this hint, that seems like a clean solution for my case:

    PYTEST_PLUGINS="pytest_lazyfixture" python3.9 -m pytest

    There is also the "-p" option which shoul early-load plugins by module
    name but it didn't work for me. The environment variable did.

    Thanks,
    Michael

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Goirand@21:1/5 to Michael Fladischer on Fri Oct 22 23:30:01 2021
    On 10/21/21 8:55 PM, Michael Fladischer wrote:
    Hi,

    I'm working on src:pytest-lazy-fixtures and was trying to get the
    unittests to run but it seems that I have run into a problem that I'm
    not sure on how to fix it in a clean way.

    pytest-lazy-fixtures is a pytest plugin and registers itself through the Python entrypoints mechanism. In its unitests, it assumes that this registration has already happened but during dh_auto_test there is no .egg-info directory available. I could use "python3 setup.py develop -x"
    to generate it, but this registers the package in /usr inside the build chroot which I think is not the best solution.

    Is there an other, less intrusive way to register the entrypoint before running the tests?

    Thanks,
    Michael


    Hi Michael,

    I do this in many of my packages (or a variation of this) to solve that problem:

    override_dh_install:
    python3 setup.py install --install-layout=deb \
    --root=$(CURDIR)/debian/tmp

    ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
    PYTHONPATH=$(CURDIR)/debian/tmp/usr/lib/python3/dist-packages \
    dh_auto_test
    endif

    dh_install

    override_dh_auto_test:
    echo "Do nothing..."

    I hope this helps,
    Cheers,

    Thomas Goirand (zigo)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry Shachnev@21:1/5 to Michael Fladischer on Sat Oct 30 22:10:01 2021
    Hi Michael!

    On Thu, Oct 21, 2021 at 08:55:51PM +0200, Michael Fladischer wrote:
    Hi,

    I'm working on src:pytest-lazy-fixtures and was trying to get the unittests to run but it seems that I have run into a problem that I'm not sure on how to fix it in a clean way.

    pytest-lazy-fixtures is a pytest plugin and registers itself through the Python entrypoints mechanism. In its unitests, it assumes that this registration has already happened but during dh_auto_test there is no .egg-info directory available. I could use "python3 setup.py develop -x" to generate it, but this registers the package in /usr inside the build chroot which I think is not the best solution.

    Is there an other, less intrusive way to register the entrypoint before running the tests?

    I am a bit late, but nobody in this thread mentioned debian/pybuild.testfiles.

    It is a list of files or directories that pybuild will copy to its build directory before running the tests, and delete after running them.

    If you have *.egg-info in the orig tarball, just add that file (and the tests directory itself) to debian/pybuild.testfiles.

    If no, also add this line to debian/rules:

    export PYBUILD_BEFORE_TEST={interpreter} setup.py egg_info

    Example package where I am doing this is pybtex-docutils.

    --
    Dmitry Shachnev

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEE5688gqe4PSusUZcLZkYmW1hrg8sFAmF9pVYACgkQZkYmW1hr g8uzQQ/+Lz/eidLPkA5eig47RtIOxsxeVzuBKQ5yc5bWmIJGDWcvUUTMjUlAeZDQ SF6rXVZzTAfhpYZRdokP9SBWzDTv+nmlln2396Uvkd6Lu5OEF/XRSqDAe5hoB1Nx OGrCILhU8psCwUYWxRhnwHPfI6wkIl3UUaH7VTbVUsc21fjzWW2kIAkIvigzwIYS a3u5omWifIOYADuDbfAXh9sKA9vpo1JXRDFQJAbJwAeF++b/P3QIRWLVPgopg6gD nb7/tEG9lEZfAcCRVmJ4jjZy7uxQQWWgJyY8kh3DwFEQwXiHqmM8Z0tFYzHJSDsO NVqrODz2iyxLgAs2jtC5bScf9Gd7LKo8rK0yiFxL0Lohy9tY3R4nm/CfUTXHduzi poXU1HDfHGizSiZczRff/P50CjylWPld6kTvjb4ZehRd8pjH7f/UG9E6dpMz+PxA 1EacQydzndWU6dLCL1CikN82gm9pbchaFdInJkPmYgHqItmvjU5fmPUSmawzmCLp qR/8X0azNEd/fIVLwX5VbM/W05o+Hy3Sl4XHOQv3Qf1H8/OJcbU0CgTMPNr4mlkp 89AQSA82wcTsuG39Sn74bLye2Rr23HYm5TWs6FeEStPna8wiJEJ31IuVG+3PH+pB vNxsowAKKC4VZ4vI9PDufwWJ6Cft6QlPS5pWdnfazdwA785sTjc=
    =5Ozp
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael Fladischer@21:1/5 to All on Sun Oct 31 14:50:01 2021
    Hi Dmitry,

    Am 30.10.2021 um 22:04 schrieb Dmitry Shachnev:
    I am a bit late, but nobody in this thread mentioned debian/pybuild.testfiles.

    It is a list of files or directories that pybuild will copy to its build directory before running the tests, and delete after running them.

    If you have *.egg-info in the orig tarball, just add that file (and the tests directory itself) to debian/pybuild.testfiles.

    thanks, I wasn't aware of this, I'll use it instead of PYBUILD_(BEFORE|AFTER)_TEST to copy test assets into place.

    If no, also add this line to debian/rules:

    export PYBUILD_BEFORE_TEST={interpreter} setup.py egg_info

    That seems like the best/most elegant solution to me. Thanks for
    sharing! So d/pybuild.testfiles is taken into account after PYBUILD_BEFORE_TEST?

    Regards,
    Michael

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry Shachnev@21:1/5 to Michael Fladischer on Sun Oct 31 15:20:02 2021
    On Sun, Oct 31, 2021 at 02:40:41PM +0100, Michael Fladischer wrote:
    If no, also add this line to debian/rules:

    export PYBUILD_BEFORE_TEST={interpreter} setup.py egg_info

    That seems like the best/most elegant solution to me. Thanks for sharing! So d/pybuild.testfiles is taken into account after PYBUILD_BEFORE_TEST?

    Yes, before.

    --
    Dmitry Shachnev

    -----BEGIN PGP SIGNATURE-----

    iQIyBAABCgAdFiEE5688gqe4PSusUZcLZkYmW1hrg8sFAmF+n8AACgkQZkYmW1hr g8vf0Q/46E6QqvUdllOOc11h/TYWrPhaOtITO1r1n8E+Tpj0G5kZJ5PYaVkm5OsL uavBJVv5yWvjrNEbcFUftV9688X/Uv2e1MglrGGTXiZm8usVno6cz3xfO34t5R3y Bm+c1AggzhgbqcSCGlydmLHXa0T44+edgAEtnt/YzAQWM6PDnliv3jV1CcRQ3tl4 RQbkjFkfvTYVtxzKya7nnaN08djBa/YCw1EPZgtV8W4KrL2CZSwVgdJhpwjgIwdp rFh/QksaBrRc6WjiLDN7Lfe9Jnn+fiuAj4filbx9q6dHR1jCZwgevTtHeGYxbIRN cqZALsMmOJORHmcuZeSQBEWQmx+ZbfNA1+DjZecYpnqgXB6QTW0UoUkyOMh/+WGc IY3GUp+UPGbjXkS8pB++zMKr21qdNJOZbLvR02jCO0zB//3Lx+gfdSB5X7380BWf cPGA9mLXDq6RosTy4w4wujhxcLJYI9pZH28DnM55UDqECw2BbuT1N4IPUcuK+ga1 73dPfU9RHneA06Ks96CPE7nB50Tl5BeMudaVYE38uwqx5XZe4YoaQ3b+VbUDkeB5 nWB1Q2Bal3KhlMm37qELHWuyQQ0IhfLG2im7KZiIbVDF6VOuss5+fM44jpA+Arru 5+KhrMrWDE+g5Y7IExvGDz+rsS8ISKoqlWQ0e2e7oAMCo2VtIQ==
    =18Kj
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)