Hi all,

When I updated pytest-mock, I noticed that pytest is somewhat out of
date and it would be good to upgrade it. But it's quite a major
package, and I don't really want to do it without a go-ahead from
others.

Perhaps we could upload a newer version to experimental first to see
what breaks?

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Julian,

On 2022-06-02 11:23, Julian Gilbey wrote:

When I updated pytest-mock, I noticed that pytest is somewhat out of
date and it would be good to upgrade it. But it's quite a major
package, and I don't really want to do it without a go-ahead from
others.

Perhaps we could upload a newer version to experimental first to see
what breaks?

I would suggest ratt-rebuilding all reverse dependencies. Could that be
done?

Best,
Andrius

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I would suggest ratt-rebuilding all reverse dependencies. Could that be
done?

there order of thousands rdeps, i dont think it's fair to ask any
individual contributor the time and resources to check that via ratt.

Something i've done in the past (f.e. with numpy and matplotlib) is
leveraging Lucas' archive rebuild infrastructure to run a rebuild the
rdeps with a new package, usually uploaded in experimental.

I think a service should be provided to developers/maintainers to test
their packages against their rdeps, but it's not there yet (i prepared
an initial plan for it, but never got to actually implement it).

Regards,
--
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Le jeudi 02 juin 2022 à 10:28 -0400, Sandro Tosi a écrit :

I would suggest ratt-rebuilding all reverse dependencies. Could
that be
done?

there order of thousands rdeps, i dont think it's fair to ask any
individual contributor the time and resources to check that via ratt.

Agreed. If the list of packages to check can't be handled by my modest
setup during the night, I won't check them all.

Something i've done in the past (f.e. with numpy and matplotlib) is leveraging Lucas' archive rebuild infrastructure to run a rebuild the
rdeps with a new package, usually uploaded in experimental.

When I upgrade a package with a number of rdeps I deem unreasonable for
my setup, I pick some rdeps semi-randomly (mostly random, but having a
look at the list to hand-pick a few promising ones, or make sure I drop problematic ones [sagemath comes to mind]).

That won't guarantee nothing will break, but does detect glaring
issues.

And indeed, uploading to experimental and giving people a head's up if
there's a thousand rdeps and you can only test 1% is important.

Cheers,

J.Puydt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, Jun 02, 2022 at 05:28:36PM +0200, julien.puydt@gmail.com wrote:

Le jeudi 02 juin 2022 à 10:28 -0400, Sandro Tosi a écrit :

I would suggest ratt-rebuilding all reverse dependencies. Could
that be
done?

there order of thousands rdeps, i dont think it's fair to ask any individual contributor the time and resources to check that via ratt.

Agreed. If the list of packages to check can't be handled by my modest
setup during the night, I won't check them all.
[...]

I believe that ci.debian.net checks packages against packages in
experimental (see
https://ci.debian.net/packages/s/spyder/unstable/amd64/ for example),
so it may be that the work is already done for us; what I don't know,
though, is how to extract this information for the package in
experimental. Perhaps worth asking the debian-ci people.

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 2022-06-03 at 19:08 +0100, Julian Gilbey wrote:

I believe that ci.debian.net checks packages against packages in
experimental (see https://ci.debian.net/packages/s/spyder/unstable/amd64/ for example),
so it may be that the work is already done for us; what I don't know,
though, is how to extract this information for the package in
experimental.  Perhaps worth asking the debian-ci people.

I think this page includes debci results for experimental:

https://release.debian.org/britney/pseudo-excuses-experimental.html

It shows what would happen when migrating experimental to unstable.

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmKaw54ACgkQMRa6Xp/6 aaOCSw/+JIGE0rSogVoCZEjNRvLImQCYdtPy29rjQviFQBRlgyg1CgQjvSBEzn3M cKoJiqbBajwgtB8ULqeHvHJ8Zk9utkoQ0xt+JjtHXWarTyr1HxxKP+A6yXEnd5zh r81qHT4PdxCN8ZCw6BqFcYwDxwc/t4zRboGU0abrWTIqqI1oeU7SdpO2KNnaDwcl t9Bu97msOb33aQzQ94Ny1WuHxyfPBKNvDr+qyVNabRF2BvZwGPF6ReVMx/197eiw bA6thDGeLfWNdwgxRYdh5SDEs6Qg50mN1rCGkNgzL8fmbBTZfVkpSjU5/KRtw8oW 7JlfiEB5Xm4mf3MQcEPom0UDxfzeInNNWepT5swjnz74AS7pen/YiamkB+pHgUji rm+TvGYTs3Nh0hmJC3gxQlPrnGoyFbnnfjBcY6YMhM7yZSvYIqUElAuh9v+dLzmb qjF9PVZyxrPxYZBAfMMvkkkS268CvtOzmVB+ksdnnvuq2lx1AANtAi5wiuvx3Dv9 srb0kUlNJCkedykJDt+7X92mDUa6LIliVVuePxLi4I/zKKJxaTdikEuf3w9cYXeb qfIFiF51e51GIh6uz+HGIsCq29uobx5qDeC8A63pC2KKgyMzgO4e+XjEgHnZEuF7 85842cpxFaNNZR+TnVIq6+v1KQBYEXkSAVGwwhJGQnz9U3h/HI4=
=GPD0
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jun 04, 2022 at 10:29:53AM +0800, Paul Wise wrote:

On Fri, 2022-06-03 at 19:08 +0100, Julian Gilbey wrote:

I believe that ci.debian.net checks packages against packages in experimental (see https://ci.debian.net/packages/s/spyder/unstable/amd64/ for example),
so it may be that the work is already done for us; what I don't know, though, is how to extract this information for the package in experimental.  Perhaps worth asking the debian-ci people.

I think this page includes debci results for experimental:

https://release.debian.org/britney/pseudo-excuses-experimental.html

It shows what would happen when migrating experimental to unstable.

Oh wow, thanks! That's perfect. So we can upload the new pytest to experimental and see what happens...

Anyone willing to go for it?

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I think this page includes debci results for experimental:

https://release.debian.org/britney/pseudo-excuses-experimental.html

It shows what would happen when migrating experimental to unstable.

Oh wow, thanks! That's perfect. So we can upload the new pytest to experimental and see what happens...

please be aware that that is a very partial view, in particular only
showing reverse dependencies with (meaningful) autopkgtests, which
means there could be hidden gigantic breakages not detected by that
page which will wreak havoc in unstable.

I would consider pytest a "core" python package, and so a complete
rdeps rebuild is appropriate, i suggest having a look at https://salsa.debian.org/lucas/collab-qa-tools/-/blob/master/modes/numpy-exp and then contacting Lucas to get access to the AWS rebuild machinery.

Anyone willing to go for it?

I thought you were volunteering for it? :) jokes aside, i think
preparing the new pytest upstream release for experimental may be the
"easiest" part of this ordeal.

Regards,
--
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2022-06-07 04:01, Sandro Tosi wrote:

I would consider pytest a "core" python package, and so a complete
rdeps rebuild is appropriate

+1. That is what I meant by suggesting ratt-rebuilding all the rdeps.

Best,
Andrius

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, Jun 06, 2022 at 09:01:37PM -0400, Sandro Tosi wrote:

I think this page includes debci results for experimental:

https://release.debian.org/britney/pseudo-excuses-experimental.html

It shows what would happen when migrating experimental to unstable.

Oh wow, thanks! That's perfect. So we can upload the new pytest to experimental and see what happens...

please be aware that that is a very partial view, in particular only
showing reverse dependencies with (meaningful) autopkgtests, which
means there could be hidden gigantic breakages not detected by that
page which will wreak havoc in unstable.

I would consider pytest a "core" python package, and so a complete
rdeps rebuild is appropriate, i suggest having a look at https://salsa.debian.org/lucas/collab-qa-tools/-/blob/master/modes/numpy-exp and then contacting Lucas to get access to the AWS rebuild machinery.

Hi Sandro,

Ah, that's a very good point. I was half-aware of it, but didn't
really think it through.

Anyone willing to go for it?

I thought you were volunteering for it? :) jokes aside, i think
preparing the new pytest upstream release for experimental may be the "easiest" part of this ordeal.

I guess it will depend on how much breakage it causes; the
deprecations and breaking changes listed at https://docs.pytest.org/en/stable/changelog.html see somewhat
"obscure", but who knows how much they're used in Debian packages?!

I wish I could help, but I only have a couple of hours per week for
Debian stuff, so this is beyond my capacity, unfortunately :-(

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, Jun 07, 2022 at 08:27:38AM +0100, Julian Gilbey wrote:

Anyone willing to go for it?

I thought you were volunteering for it? :) jokes aside, i think
preparing the new pytest upstream release for experimental may be the "easiest" part of this ordeal.

I guess it will depend on how much breakage it causes; the
deprecations and breaking changes listed at https://docs.pytest.org/en/stable/changelog.html see somewhat
"obscure", but who knows how much they're used in Debian packages?!

I wish I could help, but I only have a couple of hours per week for
Debian stuff, so this is beyond my capacity, unfortunately :-(

Actually, on reflection, with the potential size of this, it is
probably worth collaborating on; I could certainly participate.

Sandro: you managed the numpy transition, it seems. What is involved
in something like this? I would imagine something like:

(1) Upload pytest 7.x to experimental

(2) Arrange with Lucas to test build all rdeps against the
experimental version of pytest (by which I mean: all packages which
require python3-pytest as a (recursive) build-dependency)

(3) File bugs (with patches where possible) against all packages which
either FTBFS with the experimental pytest or which fail their
autopkgtest suite with the experimental pytest. Presumably these bugs
would have a usertag associated with them so they can be easily
monitored.

(4) After an appropriate time period, prepare NMUs for remaining bugs.

(5) Once all bugs are closed, upload to unstable.

I could certainly do (1) and help with (3)-(5) if someone else can do
(2) and help with (3)-(5).

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Sandro: you managed the numpy transition, it seems. What is involved
in something like this? I would imagine something like:

(1) Upload pytest 7.x to experimental

i took care of this just now, uploading pytest/7.1.2 to experimental
(and i've messed up the branches on salsa, so i've committed my
changes to `experimental2`)

(2) Arrange with Lucas to test build all rdeps against the
experimental version of pytest (by which I mean: all packages which
require python3-pytest as a (recursive) build-dependency)

I'll take care of this soon, likely after pytest has been built on a
buildd host (so will be either later today EST or tomorrow)

(3) File bugs (with patches where possible) against all packages which
either FTBFS with the experimental pytest or which fail their
autopkgtest suite with the experimental pytest. Presumably these bugs
would have a usertag associated with them so they can be easily
monitored.

that's something usually Lucas can automate, but he'll provided a set
of failed/successful logs for us to look at.

(4) After an appropriate time period, prepare NMUs for remaining bugs.

(5) Once all bugs are closed, upload to unstable.

I could certainly do (1) and help with (3)-(5) if someone else can do
(2) and help with (3)-(5).

Best wishes,

Julian

--
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, Jun 07, 2022 at 03:49:37PM -0400, Sandro Tosi wrote:

Sandro: you managed the numpy transition, it seems. What is involved
in something like this? I would imagine something like:

(1) Upload pytest 7.x to experimental

i took care of this just now, uploading pytest/7.1.2 to experimental
(and i've messed up the branches on salsa, so i've committed my
changes to `experimental2`)

Amazing, thanks! I might have used a version number such as -1~exp1,
but that is fairly immaterial. And at least I'm not the only one to
mess up things on salsa! ;-)

(2) Arrange with Lucas to test build all rdeps against the
experimental version of pytest (by which I mean: all packages which
require python3-pytest as a (recursive) build-dependency)

I'll take care of this soon, likely after pytest has been built on a
buildd host (so will be either later today EST or tomorrow)

Great!

(3) File bugs (with patches where possible) against all packages which either FTBFS with the experimental pytest or which fail their
autopkgtest suite with the experimental pytest. Presumably these bugs would have a usertag associated with them so they can be easily
monitored.

that's something usually Lucas can automate, but he'll provided a set
of failed/successful logs for us to look at.

OK, that's really helpful.

(4) After an appropriate time period, prepare NMUs for remaining bugs.

(5) Once all bugs are closed, upload to unstable.

I could certainly do (1) and help with (3)-(5) if someone else can do
(2) and help with (3)-(5).

Best wishes,

Julian

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 2022-06-08 14:47, Julian Gilbey wrote:

1. finalcif: it seems like an update to python3-gemmi has broken this
package

Yes, this is indeed the case with finalcif. Will report and investigate
it soon.

Best,
Andrius

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, Jun 07, 2022 at 03:49:37PM -0400, Sandro Tosi wrote:

Sandro: you managed the numpy transition, it seems. What is involved
in something like this? I would imagine something like:

(1) Upload pytest 7.x to experimental

i took care of this just now, uploading pytest/7.1.2 to experimental
(and i've messed up the branches on salsa, so i've committed my
changes to `experimental2`)

I've just looked at https://release.debian.org/britney/pseudo-excuses-experimental.html -
it's encouraging that there have only been a handful of failures on
amd64. There have been a few more on arm64, but they seem mostly
insignificant or transient. Here are the failing tests:

1. finalcif: it seems like an update to python3-gemmi has broken this
package

2. fpylll: The failures are happening in the current unstable version
as well.

3. glyphslib: The failures are happening in the current unstable
version as well.

4. junitparser: This looks like it might be due to the new version of
pytest.

5. jupyter-client: The arm64 failure looks likely to be transient

6. monitoring-plugins-systemd: This looks like it might be due to the
new version of pytest - it's not finding any tests.

7. nibabel: This is a new warning I haven't seen before: there's a ResourceWarning about an unclosed file, which is putting a message on
stderr and causing the test to fail.

8. pytest-pylint: This looks like it may be due to the new version of
pytest, but I'm not sure.

9. pytest-twisted: Bizarre; it's failing to find the testdir fixture;
pytest 7.x does not claim to have deprecated this, so something is
weird here.

10. python-ase: not sure why the pytest.warns call raises an error
rather than just a warning. But this is certainly a pytest 7.x issue.
I'm not sure how best to rewrite the code in these tests; different
calls are now needed in the two cases (pytest.warns(...) in the first
case, warnings.catch_warnings() in the second, perhaps?).

11. python-async-lru: this throws an error to stderr:
Future exception was never retrieved
future: <Future finished exception=ZeroDivisionError()>
ZeroDivisionError
No idea what that's about.

12. python-b2sdk: this fails to select any tests, so presumably a
pytest 7.x issue.

13. python-email-validator: interesting error on arm64, suggesting the
test needs fixing (but not pytest 7.x related)

14. python-etelemetry: looks like it is a pytest 7.x issue (change in
keywords for skipping?)

15. python-httplib2: similar to python-etelemetry

16. python-parameterized: this looks likely to be a pytest 7.x issue;
the meaning of missing_tests has perhaps changed? Would need to look
at the code more carefully to understand this.

17. python-pytest-subtests: this is a pytest 7.x issue; the package
has been updated upstream

And that's it, so it really doesn't seem too bad.

Best wishes,

Julian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)