On September 8, 2024 1:43:33 PM UTC, Stefano Rivera <stefanor@debian.org> wrote:

Hi Louis-Philippe (2022.11.28_01:46:58_+0200)

Too often, a mistake or a misconfiguration leads to no tests being detected >> when trying to run the upstream testsuite.

When this happens, the result of the test command typically looks like "Ran >> 0 tests in 0.000s".

A couple of years later:

We've implemented the feature in Python 3.12, unittest's runner now
exits with return value 5 if no tests were discovered, like pytest does. >https://github.com/python/cpython/issues/113661

We initially ignored this exit status in dh-python, to allow us to
transition to Python 3.12, without too much pain.

I've just done a rebuild test without ignoring it to see how much effort
it would take to be able to use this feature:

I built 6440 packages build-depending on dh-python in one way or
another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs.

To fix these build failures, package maintainers would have these
options:

1. Get the build to run some unit tests (assuming they exist),
2. override_dh_auto_test with something noop,
3. export PYBUILD_DISABLE=test,
4. We could make this failure opt-in in dh-python. Maybe via an explicit
--test-unittest option that selects the unittest runner. If you don't
explicitly select this runner, you'd get an attempt to run tests by
with unittest, and no failure if no tests are found.

The downside of options 2 and 3 is that if the package does start
gaining an upstream test suite, nothing ever will attempt to run it. I
think that's OK?

If you want to experiment yourself, there's a version of dh-python in >experimental that will treat no tests as an error.

Should we file bugs for these packages? Or implement option 4?

Attached is a current dd-list.

Thanks. I think some variation of #4 is the right answer. Causing a thousand packages to FTBFS isn't great. I would find it easier to have an environment variable (similar to what you suggested for #3) instead of adding an override, but that's more of
an implementation detail.

Scott K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)