TL;DR: I tried to think about what all would go in non-free-firmware if
we create it.
I think there are some complicated questions especially around source
dvds and dependencies.


Hi. So it sounds like a number of the options involve creating a non-free-firmware component, and we might even have rough consensus to do
that.

I'd like to explore what would go in that component and how that component would interact with the existing archive components.

I started by running apt search firmware and admittedly found that it
was a bit more tricky to figure out what was actually firmware than I
hoped.
But we have lists of packages presumably from the existing unofficial
non-free images.
So let's assume that at least all those packages can move to
non-free-firmware.

And for binary packages, that's probably a good starting place at least
if we don't think too hard.

Unfortunately, things get more complicated when I start thinking about
source and building, and there are even some complicated cases for
binaries.

Some Assumptions
================

1) We are making changes around firmware for practical reasons.
We are not revisiting questions like whether firmware is software in the meaning of DFSG, or whether everything other than licenses in main needs
to follow the DFSG.
Yes, some people might want to revisit those things, but for the most
part, our motivation is about helping our users by giving them images
that actually work securely on hardware they have.

2) We value being able to build from source when we can. We value being
able to have reproducible builds when we can.
We don't want to take steps backward in those areas in order to get
hardware working better.

3) We want to limit what goes into non-free-firmware so as not to
encourage people to install arbitrary non-free software.
I think the actual limits are unclear, but are no broader than non-free packages that make hardware work. I'll talk about the specific limits
later.
But I think that it is widely agreed that if we're going to create a non-free-firmware component we don't want all of non-free sneaking in.
If we didn't care about that, we could just use the existing non-free.

Build Dependencies
==================

So, many packages containing firmware are just binary blobs.
If assumption 2 is true, we don't want to require that be the case.
If partial source code is available, we want to be able to build those
parts.
If source code is available, but has restrictions (for example certain modifications are not permitted), we want to be able to use that.

So, when possible, we want to b e able to build firmware.

What happens when firmware wants to build-depend on tools in non-free?

I think we have a few options:

1) Don't actually build the firmware; ask maintainers to include build artifacts in the source package. This violates assumption 2.

2) Allow non-free-firmware to build-depend on non-free. This is fine,
and may be my preferred option. It has implications for source
distribution: effectively sources to the non-free-firmware component
would not be useful without sources to non-free. See below in the
"Source Distributions" section for why that might be problematic.

3) Move build tools needed by non-free-firmware into non-free-firmware.
That at least partially violates assumption 3 above, because we are now
making some non-free build tools available with non-free-firmware.

4) Create a non-free-firmware-build component and move the build tools
needed by non-free-firmware there.
Builds of packages in non-free-firmware are permitted to build-depend on non-free-firmware-build, but the installer would not enable non-free-firmware-build in the same situations it enables
non-free-firmware.

In options 2-4, I think it makes sense to enable build-dependencies on
contrib for packages in non-free-firmware.
In option 3, dependencies of contrib packages needed in a build would be
moved into non-free-firmware. In option 4, they would be moved into non-free-firmware-build.


As a consequence of options 2-4, non-free-firmware would typically need
to be enabled whenever non-free is enabled.
As a consequence of option 4, non-free-firmware-build would need to be
enabled whenever non-free is enabled.

Binary Dependencies
===================

It's possible that packages in non-free-firmware could depend on
non-free libraries or downloader tools or whatever. In this case I
think it's fairly clear those packages would need to move into non-free-firmware. The arguments about build-dependencies don't really
apply. Yes, doing this may partially violate assumption 3 about
limiting contents of non-free-firmware. However, if the dependency is accurate, then the libraries or downloader tools or whatever actually
need to make their way onto the system with the firmware, and additional separation doesn't appear to provide value.

I'm hoping that packages in non-free-firmware don't end up depending on
contrib (except as part of building a package in non-free-firmware).
If they do, that gets messy.

Source Distributions
====================

We know that Debian gets used by a lot of downstreams both in terms of derivative distributions and also organizations that directly consume
our media even if they customize it heavily.
We know some of these organizations care a lot about software freedom
like we do, and some of them make different trade offs between their
users and free software.
I hope we choose to support this diverse community as much as we can.

Some people who use Debian are going to only consume Debian main just as
they do today.

However it seems likely that some people will join us in accepting the practical necessity of firmware while wanting to avoid other non-free
software.
So, for example, they might well want binary media with
non-free-firmware and main.
Yet anyone who cares that much about software freedom is likely to care
about distributing complete source for a work including source for
anything it depends on.

Earlier we talked about how it would simplify how we think about
build-depends if packages in non-free-firmware could build-depend on
non-free.
If we do that, then we are effectively linking the source components
together.
To get the source for all of the build-depends in non-free-firmware you
are going to need to get the sources for non-free.

That might be okay, but we should consider the impact on people who want
to minimize the non-free software they are exposed to.

There's a related question of what components a source package can
build for.
As I understand it, a source package in main can build both binary
packages in main and contrib.

Would we permit a source package in non-free to build binaries in non-free-firmware?
(Or alternatively a source package in non-free-firmware to build
binaries in non-free?)

Allowing this would simplify situations where for example a manufacturer provides a package that includes both firmware needed for booting a
device and a non-free management tool inappropriate for
non-free-firmware.
However it would complicate things for people wanting to minimize free
software on their media.

Back to What Goes in Non-free-firmware
======================================

I will admit that I find it fairly hard to define what is firmware and
what isn't. I also find it challenging to really defend that boundary
as the boundary we're willing to cross for practical reasons.
Steve did a good job of summarizing how what it is to be firmware has
gotten more complicated over the years.
As a reminder, firmware does sometimes run on the CPU (microcode, UEFI), sometimes is software, sometimes is more like data.
(And yes, I know that the way in which microcode runs on the CPU is
different than UEFI).

I'll admit that I find it easier to think about non-free stuff that
allows people to use their hardware than strictly thinking of firmware.
As examples to consider, do we want to include the following in our
practical divergence from software freedom purity?

* High quality proprietary voices that can improve accessibility for
screen reader users

* Data for input methods that makes input easier for non-English
environments. (I don't know if this is an area where free
alternatives have caught up)

* Machine learning models for speech that would make it easier for
people who cannot easily type to use Debian.

* Proprietary Nvidia graphics drivers.

Personally I think most if not all of the above are in the same category
as firmware at least in terms of how I think about them and software
freedom.
I'd rather come up with some rules based on other categories than
firmware vs not.
Perhaps something like non-free-firmware is for packages that make it
possible to use the system or its hardware where there is no free
alternative providing comparable functionality.
Whatever it is, I'd be very appreciative if we could take some time to
come up with guidelines rather than firmware vs not. (And if it is
going to come down to firmware vs not to articulate why that's a
reasonable compromise for our users and free software).

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQSj2jRwbAdKzGY/4uAsbEw8qDeGdAUCYnrLTAAKCRAsbEw8qDeG dAjtAQDZoMrZqYmDrC8nNqsUg7iYDLCb8eUocn6wywlOnqGZuQD+KZo1sq5D4wfc 1FysoCno1t/Ncv4RoEWuuX2grVHv7wU=
=WTa1
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

❦ 10 May 2022 14:30 -06, Sam Hartman:

2) We value being able to build from source when we can. We value
being able to have reproducible builds when we can. We don't want to
take steps backward in those areas in order to get hardware working
better.

Is there any firmware that would match this? This seems like a
complication without any current application.

As a reminder, firmware does sometimes run on the CPU (microcode, UEFI), sometimes is software, sometimes is more like data.

I see microcode as a firmware for the CPU. It is loaded into the CPU and
is not executed by the CPU in the regular way (loaded in memory, PC
point to it). We do not distribute non-free UEFI implementation and I
don't see how it would help to use hardware (usually, this is shipped
with the hardware). All the firmwares we are interested in are the ones
that get loaded to a piece of hardware.

[...]

* Proprietary Nvidia graphics drivers.

Personally I think most if not all of the above are in the same category
as firmware at least in terms of how I think about them and software
freedom.

This is quite a stretch. Maybe discuss this later. Otherwise, we will
never have anything.
--
Write and test a big program in small pieces.
- The Elements of Programming Style (Kernighan & Plauger)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

"Vincent" == Vincent Bernat <bernat@debian.org> writes:

Vincent> ❦ 10 May 2022 14:30 -06, Sam Hartman:
>> 2) We value being able to build from source when we can. We value
>> being able to have reproducible builds when we can. We don't want
>> to take steps backward in those areas in order to get hardware
>> working better.

Vincent> Is there any firmware that would match this? This seems
Vincent> like a complication without any current application.

There is certainly free firmware that matches this.
I don't know about non-free firmware.
However, one of Debian's strengths is that we actually take the time to
get things right.
If we want a quick fix, let's just enable non-free.
If we want to draw a distinction, let's actually take the time to
consider what it would mean and carefully craft the definition.

If we are having trouble getting consensus on debian-devel and still
agree that it's a good idea to do, we can pick a group of people we
trust to make the decision and empower them to work through the issues.

Sweeping aside the hard issues in favor of consensus is how we end up
with issues where people feel ill-used and hurt by the process.
I hope we choose to learn from past difficult decisions and not do that
here.

--=-=-Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQSj2jRwbAdKzGY/4uAsbEw8qDeGdAUCYnr4swAKCRAsbEw8qDeG dLjrAP4kjogh7QP6Ts/BpTTrupn7dROMx8YFhZnw8HEv/Txi4AEAigaZr69UexUV jNsnZRUmwSlTPAwChXFwJa3nA8pXUwg=hwXS
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 2022-05-10 at 14:30 -0600, Sam Hartman wrote:

So let's assume that at least all those packages can move to non-free-firmware.

For backwards compatibility, I think that the firmware component is
going to need to be a subset of non-free; i.e. packages are going to
need to be *copied* not moved from non-free to the firmware component,
which means they would be available from both non-free components. 

Otherwise users who currently have firmware packages installed from
non-free will not get (security) updates unless they know about the
switch to the new component. Security updates for the CPU firmware
available in the *-microcode packages are usually very important.
There are going to be many users who don't read the firmware
announcement and don't read any release notes about the change.

I also feel like firmware is more like a section of non-free than an
entirely different type of software licensing, which making it a
completely separate component like non-free-firmware implies.

Since the component is going to need to be a subset of non-free rather
than separate to non-free, I suggest the name "non-free/firmware",
which aims to indicate to everyone who sees it that firmware is a part
of non-free that has been copied out to allow for extra separation.

If we make firmware just an extra archive section that happens to also
create an extra component, then creating it will be simple, ftp-masters
just need to change the overrides for source packages that build
firmware-* and *-microcode binary packages from whatever section they
are in now to firmware (in main) and non-free/firmware. The firmware in
main section would of course not have the extra split out component.

Build Dependencies
==================

Even lots of the free firmware in main is not built from source in
Debian, so I don't think this will be a problem for non-free firmware,
it will pretty much never have source code.

https://wiki.debian.org/Firmware/Open

The only exception is things like firmware-sof-signed, which is libre
firmware except the binaries are built and signed by Intel, so Debian
can't build the firmware binaries ourselves, unless the approach taken
with the Secure Boot shim signing is possible. First reproducibly build
the binaries, then if they match Intel's signature, attach Intel's sig.
That relies on Intel using the same cross-compiler as Debian though.

If build-deps do become a problem, the section proposal above neatly
solves this, since non-free firmware remains in non-free and can build
against packages in on non-free/contrib as usual.

Binary Dependencies
===================

It's possible that packages in non-free-firmware could depend on
non-free libraries or downloader tools or whatever.

Some of the firmware-* packages in non-free do not contain firmware,
they are only downloaders/extractors for non-redistributable firmware,
so it wouldn't surprise me if this happens.

As examples to consider, do we want to include the following in our
practical divergence from software freedom purity?

Since clearly there will always be users with install use-cases that
aren't covered by main or even main plus non-free/firmware, perhaps we
should have multiple sets of images for different audiences with
different sets of non-free things? Each of them would explain what is
non-free and the consequences of that both on the page itself and in
prompts within the image itself. The download page could then direct
the people to the right images for them; see my proposal in the other
thread for how that might work.

https://lists.debian.org/msgid-search/683a7c0e69b081aae8c46bd4027bf7537475624a.camel@debian.org

* Machine learning models for speech that would make it easier for
  people who cannot easily type to use Debian.

This is covered by Free Software, so should not need to go into the
non-free part of the archive. OTOH Debian doesn't have GPU/TPU
infrastructure so maybe can't train models for Debian main.

https://coqui.ai/ (formerly Mozilla DeepSpeech)
https://kaldi-asr.org/

Whatever it is, I'd be very appreciative if we could take some time
to come up with guidelines rather than firmware vs not.

I think this is an "I know it when I see it" type of situation and I
think we can trust the ftp-masters to set the overrides correctly so
that the packages that are firmware end up in the right place.

Since most of the packages named *firmware* *-microcode actually
contain firmware, that would be a reasonable initial list, as would
anything that puts files in the /lib/firmware/ directory.

Some of the packages (like firmware-siano) are not in any way needed by
the Debian installation process, despite containing firmware according
to reasonable definitions of that. They aren't needed for basic
functionality of a system either, just for specialised things
(receiving TV signals for eg). So they very likely aren't needed on the non-free/firmware images and thus aren't needed in non-free/firmware?

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmJ7FgUACgkQMRa6Xp/6 aaMHTQ//QQubrUICmsQD4kTtVf1UAJXLWQdm9AiS9XSh2NXJTOOtbqXUZyHMTaB9 Z5tDIanNnogCDTmIP+hzourUBlkebBkT5oRnlp1T0AOy/V+IxqGtwHnxwYiQHuwb pTsztcVmKOq4Ui94ceLsY+Fg8BYqX9N5liQ5XgBMTymYhuk/XRtpCyieYNtu1Tig 7evgFUX7u5RK/v42IrMQ+eu6+lWuzI5PEag8VKPGlLXDIvjqMaeIKOlpk4kE0w3N V7nJbrCYAg3Niz5DoSp6Bw5NbcuwBD8Im7om0P6MUe/+fjFP/H63xqfRUmIB9L3E HDV1v3NZClSPBJMgs5uC33tBRaL3wFmYrPI5qIxcWLTAkpMMlp2djHyW+EtcxXNF YS3TD/vAG+ArRKl1t1LQTmCNUlvi+2UcNX9l+Es7i+Ea5iyHBOCUHe6zQtVBksX4 6MwZgjFXo3mRZedFaPiWLCA9VTWFvUt55QV8N36oLAc8YMwJL9et3y+5/1TFNBME TQqkLQULPQkngSviX0Ld4afJKYFI4dlZ2QLmoiFf4VCGalap/FZkTTfbhMBrDEUQ sBQmspyUI4sPSCMcC8T8c3aaWmDDavxRD23VduueY+mpd5ZiUUdZ7A6dSXA6PbrG 3RB0Qb9UkYGhuuN3BRpywtSgyoK/EmR3+5W9DZKZ/lNlu9s8SMw=
=xcPk
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, May 11, 2022 at 12:04:15AM +0200, Vincent Bernat wrote:

❦ 10 May 2022 14:30 -06, Sam Hartman:

2) We value being able to build from source when we can. We value
being able to have reproducible builds when we can. We don't want to
take steps backward in those areas in order to get hardware working
better.

Is there any firmware that would match this?

yes, eg coreboot for some devices.


--
cheers,
Holger

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

A single bitcoin transaction alone consumes 621 KWh, or half a million times more energy consumption than a credit card payment. The bitcoin network annually
wastes 78 TWh (terrawatt hours) annually or the energy consumption of several *million* US households. https://twitter.com/smdiehl/status/1350869944888664064

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEuL9UE3sJ01zwJv6dCRq4VgaaqhwFAmJ7yGwACgkQCRq4Vgaa qhw5aw/+LuXALecZK/Q2rYYnAdD8u5wut0OlJG1SqEpQ4nwo7GGHLvTf83e0c/HI vysQDgT6iEcMX4mbWF4WzT94gIVO2l967a4diChOw1rtXFNJRuxEbOsFyBqCwm8B kpcWzqfnEohR46hXujLxbOxBS60W4JP7gJVnfNDRqfhiW9nTMTvVF24fsZxw84jr SU1QtT9mqiiVv7y3r30mxbSc7CuQuY3TfJoahe+sAc0Qa7i6bJ0uxhhnc3TWZnlP 0CgjjDnO2ID+ygnHInfiIWLio4SytWfVg4R0bS71HWq4Z+P0A/QKHzOBaM23J5Rc GasSdHqulsZ+LiHs1QSTqhPv1wn8MsV5lUj9r98f4RlA5e+FMDbxDwL2V

Quoting Thomas Goirand (2022-05-11 17:14:57)

For backwards compatibility, I think that the firmware component is
going to need to be a subset of non-free; i.e. packages are going to
need to be *copied* not moved from non-free to the firmware component, which means they would be available from both non-free components.

I think that's a good idea as it wouldn't break any setups. The number of packages in non-free-firmware is probably very small and the package data would not be duplicated on the mirrors anyways because non-free and non-free-firmware would both reference the same deb archives in the /pool directory, right?

A work around would be to have some automation to check if non-free is activated, and (propose to) update the sources.list automatically to add non-free-firmware. I'd prefer doing this, as having copies of the same package in both non-free and non-free-firmware is (IMO) a mess.

Maybe I'm lacking imagination but which approach would you take to do this reliably? If you go that route, then a heuristic is not enough. You must not break existing setups and you must also make sure never to get into a situation where that automation has to bail out or otherwise a system will end up without non-free-firmware even though it had non-free enabled.

What do you propose?

I'm also curious because I would like to do arbitrary machine-edits of user-supplied apt sources.list files but so far nothing worked reliably enough.

Thanks!

cheers, josch
--==============c83283306053075694=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmJ71REACgkQ8sulx4+9 g+ESeQ//ZRpH3+23XuW7lepzdJh9nMH9h6ErylNQhklu+INhOME3LYGFYHXshoN0 fTak8003RxiMj96SandQl3twhmQ6lB+fMSpVBsUpypAfXtS+id/NPHkkVM8dcIkC z5KRVKDwmtxVhc1Pg7vze9A7WBAs4GqM086vniD/TW51WKu68bTqx7rcBOzWnZru CsxVAT+6ogF6QM/WO8M26PvEGj7cvXh/9krpD0EI7Q16zBzf7KGprACmhFeEBReY 3S188vG/uTkrxI7cpddKcJfP33WcQh27adCa+MdjDpO+vxDAeqRMf5dcbf1uHjPz K+T3zQmUjhPBwt7FZ9XBt8V90U+h5jNHxE+4fpZjcAUOaOOYBttQPn77lVCWXwqS cRUesN4wLeRFyRfujl3Y6ECP8ClLYXIVx9BUYqPs6oVujtJqNDLTDrs+tHIXSjlq vF3cQF18ys8wwtvCpF2/G7SrqIZpZOY5xJQMe3sRTprMOZdjA5WstQD9sRjJeKFp tM8kq4UglNtBuyj1LJP7p+nmjNUj7QaNOoI0OyTwSEFJqZc7IJIBoJUjXfUPce3G rhhyAkOwUTTkbAS2JQOAQcdg+fXZzOmi3by3GaHqx7v7hB481ZzAKIc8/fRTXuxb OBeOZj3L12cOP4VQGFZUTOUZLyljSzXB2sPlP+PGsYnKyEvaL7o=
=vOVi
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/11/22 03:48, Paul Wise wrote:

On Tue, 2022-05-10 at 14:30 -0600, Sam Hartman wrote:

So let's assume that at least all those packages can move to
non-free-firmware.

For backwards compatibility, I think that the firmware component is
going to need to be a subset of non-free; i.e. packages are going to
need to be *copied* not moved from non-free to the firmware component,
which means they would be available from both non-free components.

A work around would be to have some automation to check if non-free is activated, and (propose to) update the sources.list automatically to add non-free-firmware. I'd prefer doing this, as having copies of the same
package in both non-free and non-free-firmware is (IMO) a mess.

Cheers,

Thomas Goirand (zigo)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, May 11, 2022 at 09:48:56AM +0800, Paul Wise wrote:

The only exception is things like firmware-sof-signed, which is libre firmware except the binaries are built and signed by Intel, so Debian
can't build the firmware binaries ourselves, unless the approach taken
with the Secure Boot shim signing is possible. First reproducibly build
the binaries, then if they match Intel's signature, attach Intel's sig.
That relies on Intel using the same cross-compiler as Debian though.

You may want to talk to people responsible for that firmware, reproducible builds sounds like an attainable goal to me.

On the other hand, an update to the compiler can make it produce different binaries, making the signature invalid. Pinning the exact version of the compiler would be unpleasant.

As examples to consider, do we want to include the following in our practical divergence from software freedom purity?

Since clearly there will always be users with install use-cases that
aren't covered by main or even main plus non-free/firmware, perhaps we
should have multiple sets of images for different audiences with
different sets of non-free things? Each of them would explain what is non-free and the consequences of that both on the page itself and in
prompts within the image itself.

I'd say that closed encrypted signed firmware that you need to load on
every boot is strictly more free than the same firmware burned into ROM.
While you lack the usual Free Software freedoms, you at least can upgrade
to whatever versions the vendor deigns to provide, downgrade to an old
version you prefer, get bug fixes including security fixes, etc. With
firmware burned into ROM the firmware stays broken.

Thus, even though that closed proprietary software continues to be a
problem, people who demand "pure" images are covering their eyes to
not see that evil, reducing their freedom in practice.

I prefer the nasty proprietary thing to be where I can see it.

It's not different from Microsoft "Secure" Boot signature we ship in main
-- a nasty thing that's required to use hardware you paid for.

Some of the packages (like firmware-siano) are not in any way needed by
the Debian installation process, despite containing firmware according
to reasonable definitions of that. They aren't needed for basic
functionality of a system either, just for specialised things
(receiving TV signals for eg). So they very likely aren't needed on the non-free/firmware images and thus aren't needed in non-free/firmware?

I don't see a reason to single out debian-installer, which is just a
special case of a live CD. We already produce multiple size varieties
of the installer (minimal, netinst, full) that pick which packages to
install.


Meow!
--
⢀⣴⠾⠻⢶⣦⠀ Aryans: split from other Indo-Europeans ~2900-2000BC → Ural →
⣾⠁⢠⠒⠀⣿⡁ Bactria → settled 2000-1000BC in northwest India. ⢿⡄⠘⠷⠚⠋⠀ Gypsies: came ~1000AD from northern India; aryan. ⠈⠳⣄⠀⠀⠀⠀ Germans: IE people who came ~2800BC to Scandinavia; not aryan.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 2022-05-11 at 19:38 +0200, Adam Borowski wrote:

You may want to talk to people responsible for that firmware, reproducible builds sounds like an attainable goal to me.

I don't have any of the hardware that supports SOF, so I'll leave that
up to the firmware-sof-signed maintainer etc. We don't have the
unsigned firmware (nor the needed cross-compiler packages) in Debian
yet and as I understand it, there is very little hardware that can use
the unsigned firmware and the Intel signed firmware is relatively easy
to package, while a reproducibly built Intel signed firmware would be
much more complex, so motivation to do this in Debian seems low.

On the other hand, an update to the compiler can make it produce different binaries, making the signature invalid.  Pinning the exact version of the compiler would be unpleasant.

Yeah, although perhaps Intel could be convinced to regularly re-build
and re-sign their official firmware binaries using the latest compiler versions. Also there are relatively few compiler updates in stable.
There are lots of distros with many different compiler builds,
so asking for lots of builds might not get any sympathy. Maybe they
could have a system to auto-bless distro-built binaries after builds.

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmJ8eZkACgkQMRa6Xp/6 aaP5ZQ//XwmpLLVjUQMCa9pmWQwTszhiWttjm8PmtFsZMIAx46HvB2y9aJo2iCGv thqHMLTV/PuAP96ZmfBI8sdLCywNYDastzqhgppKEQGpws7qfQeQ4sqgnAICn8WQ tqHPUlZUvcNroAN+FaAm8uiX31dz2mEoKifgzLgi+j3a4AK74tco0r+Ri88LO6S4 SdewV037N+ghSvw7k2axVDAHnwcKcmhr9whOm9YnWyAIHPkoylnrlSGSY5Ho+MH1 n83NYUnZMjfUao87ImKMir7893URHbSm/aAU1aR2MVptp+5k10jJtuR4ycEDxW33 Y1I75l/Ggl6x8aqOEVvmzLN9phEkdu6V0ssh+sP015wHY4XpeVCrUxJTVUU0Vr66 SHAxesxJN9uyq5VeoV90BtM5oUPczKFBt4XQrD93frxW3YW8Nx375YKbIh30PtAG f4Q4nX9djrbDkzv8QDcxBTmM1NVoOaLQb3YTnT57TM3xz1DdN/EKxpAAaUC3RX/r AP4+MrQNc8pk2+AeLPcYj36y51q9Y1mWGO1y7GW5psfR0iHDQYLWmFT8A4xsxmbk 0ZGrSpTyf1Qdb99IeOlZ2RZSQx8ykHoW/mZsYQfC6TP7hDkLnmUpe9N2nqkAz2OR JoscR2bKNaNLSUTTMgJRv6k/w25s4Qo1i4EixAGJRzz8VqvAh30=
=X3a6
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 2022-05-11 at 17:14 +0200, Thomas Goirand wrote:

A work around would be to have some automation to check if non-free is activated, and (propose to) update the sources.list automatically to add non-free-firmware.

That isn't feasible, since apt sources are managed external to Debian.

At my workplace we have $foo-apt-config* packages that manage our apt
sources, modifying them would cause us conffile prompts and that would
mean that upgrades to our $foo-apt-config* packages would get blocked,
since unattended-upgrades skips packages with conffile prompts.

Other people use configuration management systems that overwrite
modified files and probably manage their apt sources using those
systems, so Debian changes to apt sources would get removed. 

Others might be running Debian on read-only squashfs images so they
would never get the apt sources modifications needed to get firmware
from non-free, their image builds could either just fail or maybe
silently fail to install the needed firmware files.

I'd prefer doing this, as having copies of the same package in both
non-free and non-free-firmware is (IMO) a mess.

Having the same package in unstable and testing works fine, I don't
see why it would be different for non-free and non-free/firmware.

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmJ8dloACgkQMRa6Xp/6 aaPPnQ//QTC2PHfaaFVbOe/o0UU1yicSUvTOLVnkDIWVlv1NLiUilP3R91ycLkwO llg2q64DsaWQmgj36bRTLaDYWnJITomZ5TJYSYj8R9DA/BpAR/ojAUeqhHeOVCQf cHY0Zxw4TtXYNYrca0vJoTIvmNcYTgaa3qvz4Y4MV77z7u8WMQUuMQv69Wfo3ncB 3jvpPiwspSKYiNcHlQquxst7ueCRH7Uxb+P8J9Lncwwv+4ux+cCwHOSOHJCPOp35 29wzz/x8JGK3ocEbwYes+LCHIM5jp8YlFdIrbWGpkqxleM3vpaUCFSqOqVx5MxzP 3M5dMwsmMl1VkLi4K2vOvqVrD/IQjf+OXwhLtiWQh7vwGSA3EYiiHiZja4LHpH21 vuktsAYQl37VA7dWX2wqZ6wTW8JJqSRgaDphpbeeRWuGeA9cqJ+3nWgJq2rEE4oC 2unb8AzTWeKXhY3PIaO33U2EkfsgA/3cO3xyPhRHdXUTfVoZ0UjyXR71HBbt3l6O PQyV9ZBBqCWfxE8qN9eLTaO4fW8vkGI/FUtHwbiY8ElFnefvApsYJcxeMchcePzc oYP3DtVgq59Av8w8r9K6EGai9NFtne1femSo0XtuaQXNyTgi+wLgGlpSaDcAyYs+ yRpznyRPPT+91aZ++FWafSM10uLcyPmmQtU2mrSyNh8T0ND9OdA=
=dsg5
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/11/22 17:24, Johannes Schauer Marin Rodrigues wrote:

Quoting Thomas Goirand (2022-05-11 17:14:57)

For backwards compatibility, I think that the firmware component is
going to need to be a subset of non-free; i.e. packages are going to
need to be *copied* not moved from non-free to the firmware component,
which means they would be available from both non-free components.

I think that's a good idea as it wouldn't break any setups. The number of packages in non-free-firmware is probably very small and the package data would
not be duplicated on the mirrors anyways because non-free and non-free-firmware
would both reference the same deb archives in the /pool directory, right?

A work around would be to have some automation to check if non-free is
activated, and (propose to) update the sources.list automatically to add
non-free-firmware. I'd prefer doing this, as having copies of the same
package in both non-free and non-free-firmware is (IMO) a mess.

Maybe I'm lacking imagination but which approach would you take to do this reliably? If you go that route, then a heuristic is not enough. You must not break existing setups and you must also make sure never to get into a situation
where that automation has to bail out or otherwise a system will end up without
non-free-firmware even though it had non-free enabled.

What do you propose?

I'm also curious because I would like to do arbitrary machine-edits of user-supplied apt sources.list files but so far nothing worked reliably enough.

Thanks!

cheers, josch

I was thinking about some kind of debconf prompt when upgrading from an
older version of apt, of course disabled by default, that would propose
adding the non-free-firmware repo. This is safe enough, IMO, and can
also be used in the installer.

Cheers,

Thomas Goirand (zigo)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/12/22 04:52, Paul Wise wrote:

On Wed, 2022-05-11 at 17:14 +0200, Thomas Goirand wrote:

A work around would be to have some automation to check if non-free is
activated, and (propose to) update the sources.list automatically to add
non-free-firmware.

That isn't feasible, since apt sources are managed external to Debian.

At my workplace we have $foo-apt-config* packages that manage our apt sources, modifying them would cause us conffile prompts and that would
mean that upgrades to our $foo-apt-config* packages would get blocked,
since unattended-upgrades skips packages with conffile prompts.

Other people use configuration management systems that overwrite
modified files and probably manage their apt sources using those
systems, so Debian changes to apt sources would get removed.

Others might be running Debian on read-only squashfs images so they
would never get the apt sources modifications needed to get firmware
from non-free, their image builds could either just fail or maybe
silently fail to install the needed firmware files.

I'd prefer doing this, as having copies of the same package in both
non-free and non-free-firmware is (IMO) a mess.

Having the same package in unstable and testing works fine, I don't
see why it would be different for non-free and non-free/firmware.

If protected by a debconf prompt (by default, doing nothing...), all of
your remarks are going away.

Example text:

It looks like you have non-free firmware package(s) installed in your system, but the non-free-firmware repository doesn't look like present
in your sources.list. Do you wish to add a new file to /etc/apt/sources.list.d/debian-non-free-firmware.list to add this repository?

Just my 2 cents idea, hoping it helps,
Cheers,

Thomas Goirand (zigo)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 2022-05-12 at 16:56 +0200, Thomas Goirand wrote:

If protected by a debconf prompt (by default, doing nothing...), all
of your remarks are going away.

That means that people who only ever upgrade via unattended-upgrades or
other mechanisms that disable debconf/dpkg prompts etc aren't going to
see the prompt. At work we disable them even when upgrading from one
release to the next. I think that there are too many potential corner
cases in automatically updating apt sources and that it is much simpler
to subset non-free than require updates to apt sources.

For how to announce the availability of the non-free/firmware subset,
an apt hook that checks that installed packages from non-free only
include packages with firmware or microcode in their names seems like
a good idea, along with the usual info in the release notes etc.

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmJ9e3sACgkQMRa6Xp/6 aaNu5hAAtKldeR14nvv7bOzSDeTQI2SNkvLv49LwzWqnzztOpn5VgY8+gaD53G/c DaNSDfZzHERzamXLDUnUUSSH1rb7Doa13O+mPOLECGkuxtlI0NQkYrXsJtq4eL7n CHPsj1B56JB2NXuw7v/9ddY6IFyAnBd1NV+IF5XKdWVUA7tFmzTcMNVSoD7FcMJv g4EExegrF1R3zZ9/NFKwMiToUjuzbZ7FMeTKUKhiZ5ZMuzHiYQ4KjfQQqSV3wfjp WAPLpQDCOe7jBlNWk0qxH+Ou2YmtJV3MNx4BQbJvzsshLuF1sBiJM1PzEnnr2PmQ 2lHu1dUw0dh+NLEMA3HsD9pDoFlKtBZDM6kemxJZWBsCNsGpljbJHPslJepseowY 08dNq/8xFYfwD46Wj69SBlHZQ0ZcR5jCF8DbqSbGic020sJbYEzRjJu1GXDbCWzi WhgP3AFaIHOtzpubhB2jO/AiD5d2XO16u6aQWmAYh0nJ/WQ35/hkPn54hnmBv8do oDehevk/2fLnS4Zix7JZobCJZXZCSiLaNb7sXFK/bY4ppAqfVHGTgLQEeWnMawCH ulLY74TQNPKt2V1V+VU7mfZrYNuojqw6H/rfH8x+Udg0escIdk/9IXAUeG58+//6 F1+BlNisWrUYSxOumP6pMjqoyDE1SP1YC8SW8lhEXxUV0dz+LsM=
=GKoK
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/12/22 23:26, Paul Wise wrote:

On Thu, 2022-05-12 at 16:56 +0200, Thomas Goirand wrote:

If protected by a debconf prompt (by default, doing nothing...), all
of your remarks are going away.

That means that people who only ever upgrade via unattended-upgrades or
other mechanisms that disable debconf/dpkg prompts etc aren't going to
see the prompt. At work we disable them even when upgrading from one
release to the next.

Yeah, of course, but this is because you know what you're doing. If you
do, then you also probably read the release notes, don't you? :)

Here, we're trying to find a way to warn the people who don't know...

Cheers,

Thomas Goirand (zigo)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 2022-05-13 at 17:48 +0200, Thomas Goirand wrote:

Yeah, of course, but this is because you know what you're doing. If
you do, then you also probably read the release notes, don't you? :)

I haven't actually read the full release notes in recent years.

I expect a lot of experienced users don't bother to read the release
notes because upgrading from one release to the next is usually the
same procedure and usually fairly uneventful.

I often point people on Debian support channels to particular release
notes items though, like the suite name migration for security updates.
Debian often gets people who encounter errors due to that change but
haven't read the release notes so haven't done the change themselves,
and come asking us instead of reading the release notes.

Here, we're trying to find a way to warn the people who don't know...

I'm reminded that Debian has no automated release upgrade process, if
we had one then these apt sources changes/notifications could go there.

https://wiki.debian.org/AutomatedUpgrade

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmJ/BnEACgkQMRa6Xp/6 aaPpcxAAhtb5sjr518XBwuvuPu3Lt4vHo4B6Vin69VNbHx2TRwN3/eXEWtXi+R55 Ugxccfh3tyIQXKCB0hm0in+aswoc+S5iaqTWIvDGZCK4ZHH71SGCxNYDpwENIiF5 qQFfYejmPIo+utJlON1uZbACXB0ddV3PAwi5oiHPNNo9TiPPoyfX+cR5NWXu5h9k rHVIaFZHs8r+rqVCFmE+38KBvhMfQUJdWFazKq3zx5belSf+AENIlcwaoo0Uk5p/ 4mqQEkjbdmIxEEIqjpRlLLl3i3O3jYr86k9yrHD0ojrqui8Q2VnoijWHvAEgY2vv KXa1fCoH/wEpq4MoyLC09tuOLBbAfz7o3coPidFP3C193gEU/SodF39z59qqD8RD t68XgyXtVBja0Pe+1Rtz24J2YIf9aqipQw2bvZ3spCXg+bJIiWvEd30jfWaLcWJY dJRLVD2tdtcY2AyyN/ad7qdTnFZ6X4BaiE/77Rlb6UPlVXGrpRKGy5u+/Hlg9zhJ OZxkgp4IC24vF/f6uEtOFIjBy10mwfa/H4yfhpKRl4gQW01/wkU+FR/2l9LIEEJT wJLWL6uIeisQKvofLrBLORfEpScNrLLwDp3OosrYtgOsSHmdNL4pCa8i0GqLQNaD 9qwilycq6Uwc5LA1DgxlkdSEd2pMjaY1T27iwsE2XK2hi65k9kU=
=M8xv
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, May 11, 2022 at 07:38:18PM +0200, Adam Borowski wrote:

I'd say that closed encrypted signed firmware that you need to load on
every boot is strictly more free than the same firmware burned into ROM.

Not necessarily. Firmware-as-software and firmware-as-hardware are
covered by different laws. You can disclain warranty on software but not
on hardware. You can resale hardware but not software license, etc. This
has real implication.

Companies have legal incentives to move from hardware to software license, which are not to the benefit of the users.

Cheers,
--
Bill. <ballombe@debian.org>

Imagine a large red swirl here.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Moin

On Tue, May 10, 2022 at 02:30:04PM -0600, Sam Hartman wrote:

Build Dependencies
==================

As a consequence of options 2-4, non-free-firmware would typically need
to be enabled whenever non-free is enabled.

I don't understand why this is the case? Option 2 just needs non-free
for building non-free-firmware, not always.

Do we have that case that firmware needs non-free tools to build? Do we
now have just non-free or also no-source tools to build a firmware, just
to avoid shipping the binary of the firmware directly? It is not likely
that we could in any way modify such a firmware without access to many
other tools and documentation.

Binary Dependencies
===================

It's possible that packages in non-free-firmware could depend on
non-free libraries or downloader tools or whatever.

Downloaders can go to contrib already, so we would not need
non-free-firmware for that.

Bundling firmware and libraries is done only in one case as far as I
know: nvidia.

Source Distributions
====================

Yet anyone who cares that much about software freedom is likely to care
about distributing complete source for a work including source for
anything it depends on.

I usualy would say: they are free to rebuild the media without the
firmware on it.

As I understand it, a source package in main can build both binary
packages in main and contrib.

It is technically possible, but AFAIK discouraged, as it breaks several assumptions dak and the release team tooling does.

Back to What Goes in Non-free-firmware
======================================

I will admit that I find it fairly hard to define what is firmware and
what isn't. I also find it challenging to really defend that boundary
as the boundary we're willing to cross for practical reasons.
Steve did a good job of summarizing how what it is to be firmware has
gotten more complicated over the years.
As a reminder, firmware does sometimes run on the CPU (microcode, UEFI), sometimes is software, sometimes is more like data.
(And yes, I know that the way in which microcode runs on the CPU is
different than UEFI).

Sure, UEFI is a firmware. But in the past we always talked about the
firmware stuff that does _not_ run on the main CPU, but on the CPU
integrated into other hardware pieces. The same for microcode, it is
not stuff running on the main CPU.

* High quality proprietary voices that can improve accessibility for
screen reader users
* Data for input methods that makes input easier for non-English
environments. (I don't know if this is an area where free
alternatives have caught up)
* Machine learning models for speech that would make it easier for
people who cannot easily type to use Debian.

I see what you mean. But nothing of that actually enables the use of a particular hardware.

* Proprietary Nvidia graphics drivers.

This is explicitly code built and run on the main CPU, not firmware. It
also needs firmware running on the device itself.

Personally I think most if not all of the above are in the same category
as firmware at least in terms of how I think about them and software
freedom.

Actually they are not in the same category.

I'd rather come up with some rules based on other categories than
firmware vs not.
Perhaps something like non-free-firmware is for packages that make it possible to use the system or its hardware where there is no free
alternative providing comparable functionality.

But then we can't call it firmware. And this definition does not longer
fit firmware. Hardware and firmware are a unit, you need both to use
it. So it is not "no free alternative", but "integral but separate
shipped part of the hardware".

Regards,
Bastian

--
One does not thank logic.
-- Sarek, "Journey to Babel", stardate 3842.4

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 2022-05-15 at 20:53 +0200, Bastian Blank wrote:

I see what you mean.  But nothing of that actually enables the use of a particular hardware.

Enabling the use of particular hardware is a subset of the actual goal; enabling more people to use Debian at all, which the non-firmware items
Sam suggested (TTS, input, STT) are about. Folks with no/low visual
capacity require a TTS (text to speech) to be able to use Debian at
all. Folks with no/low English capacity require input methods for their languages. Folks with no ability to manipulate physical input devices
require an STT (speech to text) to be able to use Debian. Of course,
these items also benefit everyone else, who can have temporary capacity problems; like an STT is useful to people who are driving a car.

Another way to achieve enabling use of Debian at all for people who
need some parts of non-free but who don't want to use the rest of
non-free is apt pinning. Using pinning, the non-free installer could
detect packages needed by the user from non-free, enable them and
disable the rest.

Perhaps a better alternative to pinning would be to add a feature to
apt that warns about non-free packages before they are installed and
explains main vs non-free in terms of licenses and Debian support.

Of course subsetting non-free means that users can't see through apt
other subsets of non-free and this is somewhat desirable since we
prefer them to use main packages in preference to non-free packages
and only use non-free packages when really necessary. This point means
that it would be helpful to add additional subsets of non-free such as non-free/speech or non-free/input, for people who need them to use
Debian at all. Then there are people who don't want to use most of
non-free but need a few things in non-free like firmware and toolchain documentation, which would mean they would be happy to have other
non-free subsets available; non-free/firmware non-free/doc etc.

* Proprietary Nvidia graphics drivers.

This is explicitly code built and run on the main CPU, not firmware. 
It also needs firmware running on the device itself.

In case folks didn't see the news, the situation with Nvidia has
changed slightly since this thread started. For GPUs released since
about 2018, they moved more of the driver into the existing proprietary firmware that was embedded in the driver before, released the source of
the remaining Linux kernel parts under GPL/MIT licenses, while the
userspace parts remain proprietary.

https://developer.nvidia.com/blog/nvidia-releases-open-source-gpu-kernel-modules/

Presumably that firmware will go into linux-firmware.git and will now
be able to be shared with the nouveau open source driver, which will
mean nouveau will be able to do reclocking, which means there will be
much less reason to use the proprietary userspace driver, since nouveau
will be fast enough that most use-cases will be fine with it.

The eventual situation will be one proprietary signed firmware, one
upstream Linux kernel driver (probably nouveau updated based on ideas
and code from the recent code dump from nvidia) and two userspace
driver stacks, one proprietary and one from nouveau.

https://blogs.gnome.org/uraeus/2022/05/11/why-is-the-open-source-driver-release-from-nvidia-so-important-for-linux/

According to the LWN comments, the nvidia firmware is now 40MB of
signed proprietary RISC-V code that supports multiple generations of
nvidia GPUs and runs on the RISC-V microcontrollers in nvidia GPUs.

https://lwn.net/Articles/894861/

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmKBqCEACgkQMRa6Xp/6 aaOLYQ/9HfqmBkCFNk+NvNZpSkIo6/huYkwPLx6a+R2BOPpNqYP7cmIA63vqRIse KpzpiG7xYwEAijYVhAndpLwpxBBzHzewiDOKZAC/g6z5kAXd5Wr0kzAWQWxgnnwG uHJDYdWHvURcCKry9vj87tG3zsUXdITjZxmuK6M9FfEQX0tUeQ3lwH+A4Bm5z3rL xWWmOcYqIW/oYgC1Wmx0WlnfaFNXV7BxY7RJlHi8leG/ffiIp51lxgiwVPkfX+WV I5Eualdayd1PLBnfW/c1IYWc2MOiii9X7Cq78NieFbQ4jspBl3kczvKRVerE/LVC 5wvXAv+Mn0oSqUS2NKsoH1s6CLozuu9GVJQdv3iGEoCeh1w40/5mySQPcocqAv4k GJWN4SDvrtTwtBdGckhkUm37oywPlMKdmbWbX1RKLGYRf0Va9X6hLaMYFnZ0BlQc Z4VgQmvjwaKXA4eA2V6BhRzH1fXeUvgJ9WYUWsLxpXqs/VP1kXgY2j2Veh+YUKSb nw3IlsaQpzmmoqtMpb5pLcprFn+aPHo9qO9+gZOJ+2zoh0aoas2LT6SKM5Bj8Mo6 MS41YDnec7zleytwkRYSV1u7x/Z857Lzo48uSA0FQrCNFtFKMtusadk6Y9CdOIpC Y/bkO3i1Ggti+BhbCTe7rkszdL7dn66Yu764+KO3jlg9OPSvoJc=
=j2Cb
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)