Hi,

I will try to propose a new check to improve Debian Quality :-)

I'm using repology (https://repology.org) to report packages with home
page not work anymore and where I found a possible new home page.
But a lot of what I'm doing can be automatized.

When upstream abandon a home page, the Debian link can be used:
* by attackers to build a fake home page
* by person to register the old home page to have a lot of referrals
from a lot of GNU/Linux distro, for something totally different
* ...

but also, if upstream change to a new one can cause:
* outdated software in repository
* software based on outdated libraries
* software that seem not be maintained upstream
* removed software from Debian repository for the previous motivations
when, probably, in the new upstream there is the solution of all these problems.

For example, for Debian testing, you can see what packages have home
page problems:
https://repology.org/repository/debian_testing/problems

most have point to the htpp URI that is redirect to the htpps one, but a
lot do not respond anymore or have other problems (for example point to
a not more maintained repository, for example goolge code; there are
also some case when not all the certificate chain is validated or
similar issues).

If you open for a package the repology detail you can see which distro
are using with version (note: I see that sometime different distro use
the same package name for different upstream software): https://repology.org/project/jansi-native/versions

and if you go to the information tab https://repology.org/project/jansi-native/information

you can see in the "Homepage links" section what home page link all the
distro are using; where a number is in green so that URI is working and
must be checked to know if it is the new home page of that Debian package.

So, for example, in PTS (or in a bug report) can be reported to the DD:
1) that the package home page has some problem
2) a possible solution (in the repology page above)

For all packages that there isn't a possible solution can be created a
list (in the wiki, for example) and ask user help to find if there is a
new home page.

I wish that can be a good suggestion for the qa team.

I think that repology can also be used for other checks.

Ciao
Davide

--
What happened in 2013 couldn't have happened without free software
(He credited free software for his ability to help disclose the U.S. government's far-reaching surveillance projects).
Edward Snowden

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2021/03/08 08:25 PM, Jelmer Vernooij wrote:

On Mon, Mar 08, 2021 at 09:17:15PM +0100, Davide Prina wrote:

I will try to propose a new check to improve Debian Quality :-)

I'm using repology (https://repology.org) to report packages with home page not work anymore and where I found a possible new home page.
But a lot of what I'm doing can be automatized.

When upstream abandon a home page, the Debian link can be used:
* by attackers to build a fake home page
* by person to register the old home page to have a lot of referrals from a lot of GNU/Linux distro, for something totally different
* ...

but also, if upstream change to a new one can cause:
* outdated software in repository
* software based on outdated libraries
* software that seem not be maintained upstream
* removed software from Debian repository for the previous motivations when, probably, in the new upstream there is the solution of all these problems.

For example, for Debian testing, you can see what packages have home page problems:
https://repology.org/repository/debian_testing/problems

most have point to the htpp URI that is redirect to the htpps one, but a lot
do not respond anymore or have other problems (for example point to a not more maintained repository, for example goolge code; there are also some case when not all the certificate chain is validated or similar issues).

If you open for a package the repology detail you can see which distro are using with version (note: I see that sometime different distro use the same package name for different upstream software): https://repology.org/project/jansi-native/versions

and if you go to the information tab https://repology.org/project/jansi-native/information

you can see in the "Homepage links" section what home page link all the distro are using; where a number is in green so that URI is working and must
be checked to know if it is the new home page of that Debian package.

So, for example, in PTS (or in a bug report) can be reported to the DD:
1) that the package home page has some problem
2) a possible solution (in the repology page above)

For all packages that there isn't a possible solution can be created a list (in the wiki, for example) and ask user help to find if there is a new home page.

I wish that can be a good suggestion for the qa team.

I think that repology can also be used for other checks.

The Debian Janitor (through lintian-brush/upstream-ontologist) looks
at repology (as well as other data sources) to determine the Homepage field for Debian packages where it is missing.

Today, janitor/lintian-brush only sets the homepage field when it
is not set - it doesn't remove the homepage field when it is missing.
I was hoping to rely on duck (https://duck.debian.net/) to detect when
the Homepage field has gone bad, but it looks like duck is no longer maintainer :(

While the website is indeed down, one can still use duck as a standalone program to highlight dead urls in a source package. Granted that this is
not helping an archive wide QA check.

There was some talk about reviving duck.debian.net as part of
distro-tracker [1][2], unfortunatly, I haven't managed to free enough time
to start working on it (being busy on other projects ATM).

[1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=963887
[2]: https://salsa.debian.org/qa/distro-tracker/-/issues/51
--
Baptiste Beauplat - lyknode

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQt4kiVMTxdp/CJ4U4XSUsQeV3XMwUCYEaT6gAKCRAXSUsQeV3X M1YUAQD29ckSyyG7Rlw19aI9lL9k+2MEgpOo6Dt+gkscIhtdpQEA7xH+y5/XafxD aEkHSDYwK4hwhPEgqYYiWf+H5xlsRA4=
=4JSj
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, Mar 08, 2021 at 09:17:15PM +0100, Davide Prina wrote:

I will try to propose a new check to improve Debian Quality :-)

I'm using repology (https://repology.org) to report packages with home page not work anymore and where I found a possible new home page.
But a lot of what I'm doing can be automatized.

When upstream abandon a home page, the Debian link can be used:
* by attackers to build a fake home page
* by person to register the old home page to have a lot of referrals from a lot of GNU/Linux distro, for something totally different
* ...

but also, if upstream change to a new one can cause:
* outdated software in repository
* software based on outdated libraries
* software that seem not be maintained upstream
* removed software from Debian repository for the previous motivations
when, probably, in the new upstream there is the solution of all these problems.

For example, for Debian testing, you can see what packages have home page problems:
https://repology.org/repository/debian_testing/problems

most have point to the htpp URI that is redirect to the htpps one, but a lot do not respond anymore or have other problems (for example point to a not more maintained repository, for example goolge code; there are also some
case when not all the certificate chain is validated or similar issues).

If you open for a package the repology detail you can see which distro are using with version (note: I see that sometime different distro use the same package name for different upstream software): https://repology.org/project/jansi-native/versions

and if you go to the information tab https://repology.org/project/jansi-native/information

you can see in the "Homepage links" section what home page link all the distro are using; where a number is in green so that URI is working and must be checked to know if it is the new home page of that Debian package.

So, for example, in PTS (or in a bug report) can be reported to the DD:
1) that the package home page has some problem
2) a possible solution (in the repology page above)

For all packages that there isn't a possible solution can be created a list (in the wiki, for example) and ask user help to find if there is a new home page.

I wish that can be a good suggestion for the qa team.

I think that repology can also be used for other checks.

The Debian Janitor (through lintian-brush/upstream-ontologist) looks
at repology (as well as other data sources) to determine the Homepage field
for Debian packages where it is missing.

Today, janitor/lintian-brush only sets the homepage field when it
is not set - it doesn't remove the homepage field when it is missing.
I was hoping to rely on duck (https://duck.debian.net/) to detect when
the Homepage field has gone bad, but it looks like duck is no longer
maintainer :(

Jelmer

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEsjhixBXWVlpOhsvXV5wWDUyeI+gFAmBGiE0ACgkQV5wWDUye I+gXSBAApcIgFz2bPE4fW7dwu2YoCHqfJmBEkZRatXwdessegHY8FwDcVzYpWwX8 Ymd7sfVHNmQw3E+JLXK3oWrsCRXUq3GJXGnhf8Thj31TFumf2Xqz0Krt4xbfHlJv RXvGy17xu51mf8AOuaF1EoE8QE7AD5AsSPz/8vVhbhxK6Z7/EttX6XRy7BQX/nTd DkhHLvvDi42T7JoMMAETT0ZnEweopv8/wyEMKsGihErh4GVHMzquiqqfETl5Kxg0 Y4UUaGpGBPL6GO+uN3zK6fnT0Ye/zFNqCgsOP6sG46UV829xB+nLAhI3frIFIGgM ApLAopZ/+uhnxjkQrL/bN0S3W1R9FAraM2hhJH98W/xtgwp/8S2OTK+uwdtjd70O Lt5qMUosmZVES7U5j/uPkoLsqMAbC4ZhNGmx4jmw2NA/kclkBfrZpNNg2TiD2PmE dh0ifhPikItjJzSOA4jDxwZrMO3DilJQnYzmjtgYzQd0lZPAVXdTBzq4v5ZELleS 0Kwdy5dKGPpXzJopzV+q+ut2+MQDSNoaO+aLxPTuhqPvXSYxYuD767vA1ck/W8qS ZhEcUw+he38QYwK2bH9oDDdoCM+9PQkhM0FogPEIQJ2qXw3aK9NL2qEBoZDghulW 8oSRUo/D0IsGyr+2SaQJ1E0hJgjJyIpr7JVwAkCTgT4oE/We9Jc=
=+coG
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, Mar 8, 2021 at 8:17 PM Davide Prina wrote:

So, for example, in PTS (or in a bug report) can be reported to the DD:
1) that the package home page has some problem
2) a possible solution (in the repology page above)

There is already a bug report about this. In short, this isn't
possible yet because Repology does not have a way to export and link
to data about the URL problems it found on a per-distro basis. When my
distro URLs branch got rewritten by upstream and merged, they did not
merge the support for per-distro per-package problems pages.

https://bugs.debian.org/898031 https://github.com/repology/repology-updater/pull/615 https://github.com/repology/repology-updater/pull/624 https://github.com/repology/repology-webapp/issues/66 https://github.com/pabs3/repology-webapp/tree/superseded/package-urls

There is another bug about adding a link to the Repology
packages/versions pages, I've already done the packages page part of
that, see the "Other distros" link on your Debian Package Tracker
pages. The versions part needs someone who knows Django better than I
do to complete it.

https://bugs.debian.org/955335

--
bye,
pabs

https://wiki.debian.org/PaulWise

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)