Hello!

TL;DR: skip to The Question: below.

QEMU system mode is a whole-machine emulator which is able to run
virtual machines of various architectures. Here's the current list
of emulators on amd64:

total 417456
-rwxr-xr-x 1 mjt mjt 24042800 Apr 18 20:50 qemu-system-aarch64*
-rwxr-xr-x 1 mjt mjt 12696784 Apr 18 20:50 qemu-system-alpha*
-rwxr-xr-x 1 mjt mjt 21832624 Apr 18 20:50 qemu-system-arm*
-rwxr-xr-x 1 mjt mjt 7887152 Apr 18 20:50 qemu-system-avr*
-rwxr-xr-x 1 mjt mjt 8045936 Apr 18 20:50 qemu-system-cris*
-rwxr-xr-x 1 mjt mjt 12768624 Apr 18 20:50 qemu-system-hppa*
-rwxr-xr-x 1 mjt mjt 19455408 Apr 18 20:50 qemu-system-i386*
-rwxr-xr-x 1 mjt mjt 14768816 Apr 18 20:50 qemu-system-loongarch64*
-rwxr-xr-x 1 mjt mjt 9510992 Apr 18 20:50 qemu-system-m68k*
-rwxr-xr-x 1 mjt mjt 8065904 Apr 18 20:50 qemu-system-microblaze*
-rwxr-xr-x 1 mjt mjt 8065904 Apr 18 20:50 qemu-system-microblazeel*
-rwxr-xr-x 1 mjt mjt 14265072 Apr 18 20:50 qemu-system-mips*
-rwxr-xr-x 1 mjt mjt 14425424 Apr 18 20:50 qemu-system-mips64*
-rwxr-xr-x 1 mjt mjt 16435440 Apr 18 20:50 qemu-system-mips64el*
-rwxr-xr-x 1 mjt mjt 14256880 Apr 18 20:50 qemu-system-mipsel*
-rwxr-xr-x 1 mjt mjt 7907664 Apr 18 20:50 qemu-system-nios2*
-rwxr-xr-x 1 mjt mjt 12441616 Apr 18 20:50 qemu-system-or1k*
-rwxr-xr-x 1 mjt mjt 16583760 Apr 18 20:50 qemu-system-ppc*
-rwxr-xr-x 1 mjt mjt 17534000 Apr 18 20:50 qemu-system-ppc64*
-rwxr-xr-x 1 mjt mjt 16277552 Apr 18 20:50 qemu-system-riscv32*
-rwxr-xr-x 1 mjt mjt 16310288 Apr 18 20:50 qemu-system-riscv64*
-rwxr-xr-x 1 mjt mjt 7857424 Apr 18 20:50 qemu-system-rx*
-rwxr-xr-x 1 mjt mjt 10151728 Apr 18 20:50 qemu-system-s390x*
-rwxr-xr-x 1 mjt mjt 12671152 Apr 18 20:50 qemu-system-sh4*
-rwxr-xr-x 1 mjt mjt 12679344 Apr 18 20:50 qemu-system-sh4eb*
-rwxr-xr-x 1 mjt mjt 8779312 Apr 18 20:50 qemu-system-sparc*
-rwxr-xr-x 1 mjt mjt 13066576 Apr 18 20:50 qemu-system-sparc64*
-rwxr-xr-x 1 mjt mjt 7986096 Apr 18 20:50 qemu-system-tricore*
-rwxr-xr-x 1 mjt mjt 19507280 Apr 18 20:50 qemu-system-x86_64*
-rwxr-xr-x 1 mjt mjt 7819376 Apr 18 20:50 qemu-system-x86_64-microvm* -rwxr-xr-x 1 mjt mjt 16713616 Apr 18 20:50 qemu-system-xtensa*
-rwxr-xr-x 1 mjt mjt 16557808 Apr 18 20:50 qemu-system-xtensaeb*

Some years ago we had a single qemu-system package which included
all the above (actually a bit less than that, there was no arm64,
loongarch64 or microvm parts). All this stuff requires various
firmware packages to be useful, - such as various openbios variants,
u-boot, x86 firmware, slof, and a few more. Due to the large list
of dependencies, and large size of qemu-system package, I've split
it into multiple per-architecture-family subpackages such as
qemu-system-x86 and qemu-system-ppc. This was a trade-off between
large number of packages and a large package with big set of deps.
Some emulators went to qemu-system-misc subpackage (which grew
larger with time).

Later on, the external dependencies has been removed (quite some
firmware is now built from qemu source package). And qemu started
being used much more widely due to various debian properties being
added such as multiarch, cross-compilation efforts, and others.

More packages started using qemu-system in various ways. But due
to this strange split, the dependencies aren't usually right, and
it's at least difficult to make them right.

For example, qemu-system provides kvm mode on many platforms,
which does not enable CPU emulation, using native CPU and hence
native execution speed. This mode can be used, for example, for
isolation, for more extensive testing of packages and for many
other things besides the traditional VM usages.

So it looks like now it's a time to do something with all this.
In the recent upload (to -experimental due to freeze), I've added
debian naming for packages and binaries, for example there's now qemu-system-amd64 which is a symlink to qemu-system-x86_64 - this
way it's possible to depend or use qemu-system-${DEB_HOST_ARCH}
directly without having a translation table. It should already
help some usages to find the right package and the right binary.

The Question: how to rearrange the qemu-system-foo packages best?

I'm thinking about having two packages here instead of many:
it is qemu-system-native and qemu-system-emu. The first one
will have qemu-system binaries for the native CPU/arch, like
what qemu-system-x86 provides for x86/amd64, or qemu-system-ppc
for powerpc/ppc64 platform, also like ubuntu-only qemu-system-s390x
provides on s390x. And the other to contain all the emulators
for all other platforms, and depend on -native for various
common modules and whatnot.

Note that -native does not necessary have kvm mode, since it is
not available on all platforms, - so for some (eg sparc) it
will still be emulated. So maybe it shouldn't actually exist
on those platforms.

Another variation is to have qemu-system-kvm package which
provides native kvm-accelerated mode, plus qemu-system-emu
package with anything else. qemu-system-kvm will not be available
at all on platforms not supporting kvm (or we'll have a dummy
package there).

And the 3rd variant is to just merge everything back into single
qemu-system package the way it has been a few years back (now
with greatly reduced set of dependencies but with grown size).

This is - hopefully - the long-term goal, because upstream is
slowly thinking about building qemu-system as a single binary
which provides emulation of everything available, - obviously
we won't try to split such binary into multiple packages :)
But this goal seems to be too far in the future to be plannable.

Without such merge into single binary, the package will be large
(see full list of emulator binaries above), but hdd space is
much cheaper nowadays..

What do you think?

/mjt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Apr 19, 2023 at 8:47 AM Michael Tokarev <mjt@tls.msk.ru> wrote:

Hello!

TL;DR: skip to The Question: below.

QEMU system mode is a whole-machine emulator which is able to run
virtual machines of various architectures. Here's the current list
of emulators on amd64:

total 417456
-rwxr-xr-x 1 mjt mjt 24042800 Apr 18 20:50 qemu-system-aarch64*
-rwxr-xr-x 1 mjt mjt 12696784 Apr 18 20:50 qemu-system-alpha*
-rwxr-xr-x 1 mjt mjt 21832624 Apr 18 20:50 qemu-system-arm*
-rwxr-xr-x 1 mjt mjt 7887152 Apr 18 20:50 qemu-system-avr*
-rwxr-xr-x 1 mjt mjt 8045936 Apr 18 20:50 qemu-system-cris*
-rwxr-xr-x 1 mjt mjt 12768624 Apr 18 20:50 qemu-system-hppa*
-rwxr-xr-x 1 mjt mjt 19455408 Apr 18 20:50 qemu-system-i386*
-rwxr-xr-x 1 mjt mjt 14768816 Apr 18 20:50 qemu-system-loongarch64* -rwxr-xr-x 1 mjt mjt 9510992 Apr 18 20:50 qemu-system-m68k*
-rwxr-xr-x 1 mjt mjt 8065904 Apr 18 20:50 qemu-system-microblaze*
-rwxr-xr-x 1 mjt mjt 8065904 Apr 18 20:50 qemu-system-microblazeel* -rwxr-xr-x 1 mjt mjt 14265072 Apr 18 20:50 qemu-system-mips*
-rwxr-xr-x 1 mjt mjt 14425424 Apr 18 20:50 qemu-system-mips64*
-rwxr-xr-x 1 mjt mjt 16435440 Apr 18 20:50 qemu-system-mips64el*
-rwxr-xr-x 1 mjt mjt 14256880 Apr 18 20:50 qemu-system-mipsel*
-rwxr-xr-x 1 mjt mjt 7907664 Apr 18 20:50 qemu-system-nios2*
-rwxr-xr-x 1 mjt mjt 12441616 Apr 18 20:50 qemu-system-or1k*
-rwxr-xr-x 1 mjt mjt 16583760 Apr 18 20:50 qemu-system-ppc*
-rwxr-xr-x 1 mjt mjt 17534000 Apr 18 20:50 qemu-system-ppc64*
-rwxr-xr-x 1 mjt mjt 16277552 Apr 18 20:50 qemu-system-riscv32*
-rwxr-xr-x 1 mjt mjt 16310288 Apr 18 20:50 qemu-system-riscv64*
-rwxr-xr-x 1 mjt mjt 7857424 Apr 18 20:50 qemu-system-rx*
-rwxr-xr-x 1 mjt mjt 10151728 Apr 18 20:50 qemu-system-s390x*
-rwxr-xr-x 1 mjt mjt 12671152 Apr 18 20:50 qemu-system-sh4*
-rwxr-xr-x 1 mjt mjt 12679344 Apr 18 20:50 qemu-system-sh4eb*
-rwxr-xr-x 1 mjt mjt 8779312 Apr 18 20:50 qemu-system-sparc*
-rwxr-xr-x 1 mjt mjt 13066576 Apr 18 20:50 qemu-system-sparc64*
-rwxr-xr-x 1 mjt mjt 7986096 Apr 18 20:50 qemu-system-tricore*
-rwxr-xr-x 1 mjt mjt 19507280 Apr 18 20:50 qemu-system-x86_64*
-rwxr-xr-x 1 mjt mjt 7819376 Apr 18 20:50 qemu-system-x86_64-microvm* -rwxr-xr-x 1 mjt mjt 16713616 Apr 18 20:50 qemu-system-xtensa*
-rwxr-xr-x 1 mjt mjt 16557808 Apr 18 20:50 qemu-system-xtensaeb*

Some years ago we had a single qemu-system package which included
all the above (actually a bit less than that, there was no arm64,
loongarch64 or microvm parts). All this stuff requires various
firmware packages to be useful, - such as various openbios variants,
u-boot, x86 firmware, slof, and a few more. Due to the large list
of dependencies, and large size of qemu-system package, I've split
it into multiple per-architecture-family subpackages such as
qemu-system-x86 and qemu-system-ppc. This was a trade-off between
large number of packages and a large package with big set of deps.
Some emulators went to qemu-system-misc subpackage (which grew
larger with time).

Later on, the external dependencies has been removed (quite some
firmware is now built from qemu source package). And qemu started
being used much more widely due to various debian properties being
added such as multiarch, cross-compilation efforts, and others.

More packages started using qemu-system in various ways. But due
to this strange split, the dependencies aren't usually right, and
it's at least difficult to make them right.

For example, qemu-system provides kvm mode on many platforms,
which does not enable CPU emulation, using native CPU and hence
native execution speed. This mode can be used, for example, for
isolation, for more extensive testing of packages and for many
other things besides the traditional VM usages.

So it looks like now it's a time to do something with all this.
In the recent upload (to -experimental due to freeze), I've added
debian naming for packages and binaries, for example there's now qemu-system-amd64 which is a symlink to qemu-system-x86_64 - this
way it's possible to depend or use qemu-system-${DEB_HOST_ARCH}
directly without having a translation table. It should already
help some usages to find the right package and the right binary.

The Question: how to rearrange the qemu-system-foo packages best?

I'm thinking about having two packages here instead of many:
it is qemu-system-native and qemu-system-emu. The first one
will have qemu-system binaries for the native CPU/arch, like
what qemu-system-x86 provides for x86/amd64, or qemu-system-ppc
for powerpc/ppc64 platform, also like ubuntu-only qemu-system-s390x
provides on s390x. And the other to contain all the emulators
for all other platforms, and depend on -native for various
common modules and whatnot.

Note that -native does not necessary have kvm mode, since it is
not available on all platforms, - so for some (eg sparc) it
will still be emulated. So maybe it shouldn't actually exist
on those platforms.

Another variation is to have qemu-system-kvm package which
provides native kvm-accelerated mode, plus qemu-system-emu
package with anything else. qemu-system-kvm will not be available
at all on platforms not supporting kvm (or we'll have a dummy
package there).

In terms of naming/usage I like this suggestion of having a
qemu-system-native which would give you the best possible (even if -
as you outlined - that sometimes still is emulation) native experience
and OTOH qemu-system-emu for all the emulators.
Among other things that allows us to just always use -native in tests
where staying smaller (instead of pulling all of -emu) is helpful for
speedup and bandwidth.

But while I see the appeal of reunifying I also have often liked (and
seen people appreciate and use) this being split and explicit.
Do you consider your proposed qemu-system-native be a meta package
that depends on the right qemu-system-$arch, or an actual package with
content.

So would it be:
a)
x86: qemu-system-native -dep-> qemu-system-x86_64
ppc: qemu-system-native -dep-> qemu-system-ppc
or
b)
x86: qemu-system-native
ppc: qemu-system-native

I personally like (a) more to simplify dependencies while not breaking habits. But that might be just me, so I wanted to ask for clarification so
everyone is on the same page here.

And the 3rd variant is to just merge everything back into single
qemu-system package the way it has been a few years back (now
with greatly reduced set of dependencies but with grown size).

This is - hopefully - the long-term goal, because upstream is
slowly thinking about building qemu-system as a single binary
which provides emulation of everything available, - obviously
we won't try to split such binary into multiple packages :)
But this goal seems to be too far in the future to be plannable.

Without such merge into single binary, the package will be large
(see full list of emulator binaries above), but hdd space is
much cheaper nowadays..

What do you think?

/mjt

--
Christian Ehrhardt
Senior Staff Engineer, Ubuntu Server
Canonical Ltd

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

19.04.2023 11:41, Christian Ehrhardt пишет:

On Wed, Apr 19, 2023 at 8:47 AM Michael Tokarev <mjt@tls.msk.ru> wrote:

..

But while I see the appeal of reunifying I also have often liked (and
seen people appreciate and use) this being split and explicit.
Do you consider your proposed qemu-system-native be a meta package
that depends on the right qemu-system-$arch, or an actual package with content.

So would it be:
a)
x86: qemu-system-native -dep-> qemu-system-x86_64
ppc: qemu-system-native -dep-> qemu-system-ppc
or
b)
x86: qemu-system-native
ppc: qemu-system-native

I thought about the b) case, not a). kvm is the most interesting
use case (esp. on x86), and that's your -s390x package is about,
too, - so native-kvm qemu is free from the rest of -emu parts.

The idea is to have two binay packages, like -native and -emu,
with -native being for this architecture, with the rest being
in -emu. This way we don't have many packages, but still will
be able to optimize for the most important and often use case.

Splitting it all into individual qemu-system-x86-64, qemu-system-i386, qemu-system-loongson64 etc (so -native becomes just a Provides or
a dummy dep-only pkg) seems to be too much. I dislike so many small
packages, and multiple binaries in the same .tar compresses way
better :)

I personally like (a) more to simplify dependencies while not breaking habits.
But that might be just me, so I wanted to ask for clarification so
everyone is on the same page here.

The simplest way with deps is to merge everything into a single
qemu-system. But it is somewhat huge. Maybe we'll be able to
shrink it quite a bit by using a common shared lib for common
parts (large portion in the executables is the same), but that
requires quite some work. And this is definitely a way to go
if upstream will merge whole thing into a single binary.

Speaking of deps simplification, in 8.0 I provided qemu-system-${DEB_HOST_ARCH} variants for all qemu-system-foo packages (I think I mentioned this
already).

I also want to provide qemu-system-for-host (meta)package, to
simplify build-deps for cross-compilation cases, - that aim to
provide build-native qemu-system executable for the build-host
target (it is not the same as qemu-system-native mentioned above).

/mjt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)