On Thu, Jul 27, 2023 at 04:52:29PM +0100, Martin McCarthy wrote:

Dear Debian team,

I'm very pleased to be involved in the Debian project. Thank you for having me. Let me introduce myself. I'm Martin, I am a developer based in
Manchester in the UK and I've recently decided to dive head first into the wonderful world of FOSS. I have been in the Microsoft ecosystem for the best part of two decades and only ever dabbled in Linux and Debian a little bit during that time. I recently took part in the Debian 12.1 ISO release event and I had fun, learned a lot of things and met some interesting people (including the former DPL Sledge). I had some suggestions based on my experience that I would like to share with you.

2) Recognition of testers. Although the small group over on #debian-cd on OFTC were thankful for everyone's contributions including my own, it would
be really great for new testers like myself to have some kind of formal project recognition. It may seem like a small thing but it makes a huge difference to me personally. There are several ways of recognising the testers...some easy some not so easy. Firstly, when publishing the release notes, naming the testers (by their chosen names) that helped test the
images before publication would be a nice way to recognise us. Also, as cheesy as this may seem or sound, awarding a certificate to recognise that that person has participated in the Debian release testing process. Doesn't have to be printed of course, it could be designed in Gimp or whathaveyou. I have mocked a design which you can view here - > https://justpaste.it/de4hd (NB: this is a heads up. This link is SFW...if you happen to be in the
office right now).

If it helps, you're now a Debian contributor by virtue of having edited the wiki and done some testing.

3) Overhaul of the wiki testing process. Rattus proposed some overhauling of the ISO testing situation with the wiki. I am happy to work on that with you Rattus if you get time and we have something where we don't need to wait for the wiki to be released from another user. Although, I'm also happy to keep it as is if that's what the community wants. If it's a nod to times of old, I'm all for that too. I found the process quaint and reminded me of the old days of locking Excel workbooks. :D

Thanks for reading this, if you want to chat, I'm in various channels on Libera and OFTC on IRC or you can respond to this mail. :)

Thanks and regards,
Martin McCarthy.
==================================================
Remember when you were young? You shone like the sun.

Breathe, breathe in the air...don't be afraid to care.
Leave, but don't leave me...look around, choose your own ground. ==================================================

amacater on IRC :)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Martin!

On Thu, Jul 27, 2023 at 04:52:29PM +0100, Martin McCarthy wrote:

I'm very pleased to be involved in the Debian project. Thank you for having >me. Let me introduce myself. I'm Martin, I am a developer based in Manchester >in the UK and I've recently decided to dive head first into the wonderful >world of FOSS. I have been in the Microsoft ecosystem for the best part of >two decades and only ever dabbled in Linux and Debian a little bit during >that time. I recently took part in the Debian 12.1 ISO release event and I >had fun, learned a lot of things and met some interesting people (including >the former DPL Sledge). I had some suggestions based on my experience that I >would like to share with you.

1) When I was conducting testing of Debian 12.1, I had an idea of writing a >"Testing Wizard" Bash/Perl/Python script to help with the testing process. A >simple shell script that will test the ISO against a predefined set of tests >and gather pertinent information (such as hardware information for the team). >The tester would run the script, all the automatic code would execute (such >as gathering hardware information, ISO name, boot time, etc) then the wizard >would guide the tester through a series of required tests and then possibly >some random ones thrown in (in order to increase chances of catching bugs >before publishing). I'd be interested to hear other people's opinions on it >and possibly we could have a session to discuss it in a bit more detail if >you're interested. If I'm overthinking it or over egging the pudding here, I >am also interested to hear your feedback there too. :-)

You're describing something that overlaps quite a bit with what we've
discussed already (your point 3 below). We'd like to have a system
set up to guide testing and help us test most effectively:

* minimise the number of tests we need to cover all the features
* track what hardware people are using
* lead people through the tests if needed, to make things easier

2) Recognition of testers. Although the small group over on #debian-cd on >OFTC were thankful for everyone's contributions including my own, it would be >really great for new testers like myself to have some kind of formal project >recognition. It may seem like a small thing but it makes a huge difference to >me personally. There are several ways of recognising the testers...some easy >some not so easy. Firstly, when publishing the release notes, naming the >testers (by their chosen names) that helped test the images before >publication would be a nice way to recognise us. Also, as cheesy as this may >seem or sound, awarding a certificate to recognise that that person has >participated in the Debian release testing process. Doesn't have to be >printed of course, it could be designed in Gimp or whathaveyou. I have mocked >a design which you can view here - > https://justpaste.it/de4hd (NB: this is >a heads up. This link is SFW...if you happen to be in the office right now).

As Andy C already mentioned, we're already keen to track and recognise
the efforts of volunteers in Debian. If you look at

https://contributors.debian.org/

you'll see that we track what people are doing in a number of ways. If
you follow the link to

https://contributors.debian.org/source/Release%20image%20testers/

that's the set of data for people who've helped test our images. For
you to appear there too, you just need to set up an account and log in
to claim your data. The data is in the system, we just deliberately do
*not* publically list people who haven't signed up. Privacy,
etc... :-)

3) Overhaul of the wiki testing process. Rattus proposed some overhauling of >the ISO testing situation with the wiki. I am happy to work on that with you >Rattus if you get time and we have something where we don't need to wait for >the wiki to be released from another user. Although, I'm also happy to keep >it as is if that's what the community wants. If it's a nod to times of old, >I'm all for that too. I found the process quaint and reminded me of the old >days of locking Excel workbooks. :D

*grin*

4) Automation. I read a thread on this mailing list recently talking about >openQA and whilst I am not opposed to automatic testing by machines, I would >insist that the manual testers are still retained as relying solely on >machines for QA can be problematic (machines are not perfect just like >humans). Not only that, the testers over on #debian-cd is a community and >having full automation would mean that community no longer meets regularly to >test stuff...and thus the community disperses and fades away over time. FOSS >to me is all about community so I would hope that openQA works alongside the >ISO testers rather than replace us. This is slightly off-topic but AI is >doing a lot of damage to corporate tech and I hope that the FOSS projects >don't fall victim to it too.

ACK.

--
Steve McIntyre, Cambridge, UK. steve@einval.com < liw> everything I know about UK hotels I learned from "Fawlty Towers"

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Martin McCarthy <martin.c.mccarthy@outlook.com> writes:

4) Automation. I read a thread on this mailing list recently talking
about openQA and whilst I am not opposed to automatic testing by
machines, I would insist that the manual testers are still retained as relying solely on machines for QA can be problematic (machines are not perfect just like humans). Not only that, the testers over on #debian-cd
is a community and having full automation would mean that community no longer meets regularly to test stuff...and thus the community disperses
and fades away over time. FOSS to me is all about community so I would
hope that openQA works alongside the ISO testers rather than replace us. This is slightly off-topic but AI is doing a lot of damage to corporate
tech and I hope that the FOSS projects don't fall victim to it too.

I don't think there's any danger of OpenQA doing this (speaking as the
person who set openqa.debian.net up) because the by-hand testing and
openQA are complementary rather than competing. They're aimed at
differing targets.

The automated tests are mostly about catching regressions, mostly during development, and I'd hope that as the salsa integration comes together
they'll become something that can provide confidence to people as they contribute to D-I.

My perception is that the by-hand testing is more about making sure
things are working across as wide a set of use cases on as wide a range
of hardware as possible, and is focussed around release time.

Writing automated tests that can notice things looking "weird" is rather
hard -- OpenQA is very good at noticing that someone changed the font
kerning very slightly (which is where many of our false positives come
from), but if something is broken on a bit of the screen that's not
specified as something to look out for in the test, it'll never notice.

Also, testing on real hardware is not exactly trivial with OpenQA, so
that has not been a priority, but even if we do get to the point where
we can do openQA testing of real hardware, I wouldn't expect that the
spectrum of hardware under test would be anywhere near as wide as
currently gets tested by hand, nor would I expect for it to be useful
(or even possible) to automate all of the tests done by hand.

Of course, if fleshing out the set of tests that OpenQA runs (on VM or
real hardware) renders some of the by-hand tests mostly redundant, that
doesn't seem terrible, especially if we concentrate on automating the
tests that are particularly tedious to do by hand. (suggestions welcome :-) )

Cheers, Phil.
--
Philip Hands -- https://hands.com/~phil

--=-=-Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE3/FBWs4yJ/zyBwfW0EujoAEl1cAFAmTGaicACgkQ0EujoAEl 1cDKCg//f/Qvs0WD1ufBeFlUVhj0B0nUzCZ4Fh4pmbvz4a8J8wpeY/7ZTLKKAqvR Elfil5wIVNbbq4TGOIuBxHgnW/Bvv6CIgkPvSXR07YJymAwhApOdhM4hqNMia9X9 CAKvdZQ0aMDcPYGhVDFejXsTa1l+O/AoVH7fKpF/dqJsGzWjY91Kxwd4je58ZDOh HMQmH8clrm+4IaXQF7izzjYqtTuEtPp1UiYQUWPFP6s56qAxkgvP0naSO5VoIQhv bgrqjL686Wf7maCz6RvcvH28XsGFjUhAcIjwgVY5epjjx303aNEoCkCsBdMGQ2pU LILLSfWyTlNR9hj4tCvq+N9X6vuiQQg5QUW5af2bKmVqLwFQBt6V00P6oXqX9bKF EVX3uU/Fh3FHCTP+mcTHbM/hUPQXanqCfoZxpque1BXUdlfaVdnlX18VrT3VhYiV YGTbJ7vdmuE9Uj7kiuVc0wxRGmfTcey7SMqcab5kDPZpUQI5vSBqpVzyLfPGpx51 1AB5t6/D8/B/5ieJbsRJWwK5FvkVh94zoVYQpJ4sbPvg7cnU/L//bXISfpaKqYZI e5VAgqt/ug31lEH8xum6FQAOieRRGlX2GfeCokolHA6mEqe2i//FFHEdi0HdCoMj O9rbGsbJwHS8KCZ0hAn7A2clhhlT28K+rlQIEwXGItc64JNuzbw=ru0y
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gatewa