Can one use physical partitions one multi-boots from in Bhyve in FreeBSD
UNIX and keep multi-booting? I'd like to use Slackware GNU/Linux (and
other OS) in Bhyve but still keep it on its partition.
You can pass drives/partitions to bhyve just as you would image files. Passing -s 2,ahci-hd,/dev/nvd0p1 to bhyve makes the first partition
on my NVME show up as a block device in the VM. I have never tried
it myself with an existing installation, but I think it should be
possible.
If only passing partitions through I suspect you could have some
issues with the bootloader since each partition becomes its own block
device. You might have to either adjust the bootloader's settings or
create a separate bootloader for that VM only.
On 12/21/23 03:54, Andreas Kempe wrote:
You can pass drives/partitions to bhyve just as you would image files.
Passing -s 2,ahci-hd,/dev/nvd0p1 to bhyve makes the first partition
on my NVME show up as a block device in the VM. I have never tried
it myself with an existing installation, but I think it should be
possible.
The problem that I've had doing things like this -- admittedly with
other hypervisors and other OSs -- is that it a partition is not an
entire disk.
Thus when the VM thinks it's working with an entire disk and slices and partitions therein, it's actually doing it inside of another partition
as seen from the outside of the VM.
To pull this off I've had to pass the entire drive to the VM so that
it's inside view of the drive / block device matches the outside view of
the drive / block device.
If only passing partitions through I suspect you could have some
issues with the bootloader since each partition becomes its own block
device. You might have to either adjust the bootloader's settings or
create a separate bootloader for that VM only.
IMHO the boot loader issue is a second order problem based on the first
order problem of nesting (slice & partitions inside a slice & partition).
What kind of issues have you been having? When I pass a partition
through, it appears as a block device without a partition table with
a file system directly on the block device.
Once Linux/FreeBSD is booted, I have never had an issue mounting and
using the storage, ex. mount /dev/sda /mnt inside a Linux VM.
That said, in the scenario where you multi-boot, passing the entire
drive and selecting the right OS in the boot loader is probably the
easiest solution, but I imagine things can go horribly wrong if you accidentally boot from the host's drive.
Only tangentially related to the matter at hand, but I usually forego
the partition table on things like USB sticks when I only want a
single file system and have never had an issue with it.
Considering I've never tried doing this, the bootloader issue was
mostly me speculating because it was really the only thing I could
imagine having issues with considering both FreeBSD/Linux, in my
experience, handle file systems directly on block devices just fine,
but I don't know what Grub or other common bootloaders think of
booting a drive without a partition table.
On 12/21/23 10:14, Andreas Kempe wrote:
Once Linux/FreeBSD is booted, I have never had an issue mounting and
using the storage, ex. mount /dev/sda /mnt inside a Linux VM.
Even if it works, which Linux and FreeBSD probably can / will, it is an atypical arrangement to use a file system on a raw partitioned block device.
What's more is that is different behavior inside vs outside of the virtualization. Ergo inconsistent behavior. Now you are dependent on
an OS that can tolerate the inconsistency.
There's also a decent chance that the actual device names will be
different inside vs outside. How big of a problem that is will be
dependent on the OS.
I'll say it this way, the difference in accessing the block device
inside vs outside will be different and non-transparent.
That said, in the scenario where you multi-boot, passing the entire
drive and selecting the right OS in the boot loader is probably the
easiest solution, but I imagine things can go horribly wrong if you
accidentally boot from the host's drive.
I would suggest a different drive to avoid the issue that you mention.
Can one use physical partitions one multi-boots from in Bhyve in FreeBSD
UNIX and keep multi-booting? I'd like to use Slackware GNU/Linux (and
other OS) in Bhyve but still keep it on its partition.
Den 2023-12-21 skrev David Chmelik <dchmelik@gmail.com>:mdconfig -u 0 -d
Can one use physical partitions one multi-boots from in Bhyve in FreeBSD
UNIX and keep multi-booting? I'd like to use Slackware GNU/Linux (and
other OS) in Bhyve but still keep it on its partition.
To not just talk out of my ass, I tried this with my Gentoo
installation and got it to work.
Create an image to use for a custom EFI:
dd if=/dev/zero of=/path/to/efi.img
mdconfig /path/to/efi.img
gpart create -s GPT md0
gpart add -t efi md0
newfs_msdos /dev/md0p1
Boot the VM using a live CD medium:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 3,ahci-cd,/path/to/install-amd64-minimal-20231210T170356Z.iso \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
The iso is the Gentoo minimal install ISO. /path/to/efi.img is the
custom EFI file system created earlier and /dev/gentoop2 is the
partition where the Gentoo install resides.
When the VM boots, /dev/sdb is the gentoo file system and /dev/sda1 is
the EFI partition. Mount the file systems and chroot.
mount /dev/sdb /mnt/gentoo
mount /dev/sda1 /mnt/gentoo/efi
arch-chroot /mnt/gentoo
In the chroot, backup the old grub config and generate a new one:
cp -r /boot/grub /root
grub-install --target=x86_64-efi --efi-directory=/efi --removable grub-mkconfig -o /boot/grub/grub.cfg
I did not need to adjust the fstab since my fstab was empty. Make sure
to check it if you have mounts in there.
Exit the chroot and shut down the VM:
exit
shutdown -hP now
The VM can now be booted without the installation medium and boots
just fine:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
It is doable (since I did it), but how easy it is will depend on your distribution. I have not used Slackware so I am afraid I don't really
know how easily the equivalent of what I did for my Gentoo
installation would be.
A very good point. I only thought about the root device, which
for Linux is passed by the bootloader so in my mind that probably
wouldn't be an issue if one got the bootloader working.
If you have any additional storage set up in the fstab, you will need
to modify it for use as a VM.
Naturally. I was writing with David Chmelik's scenario in mind with
a multi-boot and partitions.
To not just talk out of my ass, I tried this with my Gentoo
installation and got it to work.
Create an image to use for a custom EFI:
dd if=/dev/zero of=/path/to/efi.img
mdconfig /path/to/efi.img
gpart create -s GPT md0
gpart add -t efi md0
newfs_msdos /dev/md0p1
Boot the VM using a live CD medium:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 3,ahci-cd,/path/to/install-amd64-minimal-20231210T170356Z.iso \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
The iso is the Gentoo minimal install ISO. /path/to/efi.img is the
custom EFI file system created earlier and /dev/gentoop2 is the
partition where the Gentoo install resides.
When the VM boots, /dev/sdb is the gentoo file system and /dev/sda1 is
the EFI partition. Mount the file systems and chroot.
mount /dev/sdb /mnt/gentoo
mount /dev/sda1 /mnt/gentoo/efi
arch-chroot /mnt/gentoo
In the chroot, backup the old grub config and generate a new one:
cp -r /boot/grub /root
grub-install --target=x86_64-efi --efi-directory=/efi --removable grub-mkconfig -o /boot/grub/grub.cfg
I did not need to adjust the fstab since my fstab was empty. Make sure
to check it if you have mounts in there.
Exit the chroot and shut down the VM:
exit
shutdown -hP now
The VM can now be booted without the installation medium and boots
just fine:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
It is doable (since I did it), but how easy it is will depend on your distribution.
I have not used Slackware so I am afraid I don't really know how easily
the equivalent of what I did for my Gentoo installation would be.
Pre-script: I'm sick and tired of the UI->UX changes made in
contemporary Thunderbird. Different buttons must now be used between
email and news to direct replies where I want them to go. >:-|
On 12/22/23 05:41, Andreas Kempe wrote:
To not just talk out of my ass, I tried this with my Gentoo
installation and got it to work.
Testing is good.
Aside: Are you running Gentoo (user space) on top of a FreeBSD kernel?
-- I believe it's an officially supported configuration, but I think
that most will assume Linux in association with Gentoo. The "bhyve" and "mdconfig" seem very *BSD to me.
Create an image to use for a custom EFI:
dd if=/dev/zero of=/path/to/efi.img
mdconfig /path/to/efi.img
gpart create -s GPT md0
gpart add -t efi md0
newfs_msdos /dev/md0p1
Boot the VM using a live CD medium:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 3,ahci-cd,/path/to/install-amd64-minimal-20231210T170356Z.iso \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
Would you please break down what that command is doing, what virtual
hardware it's creating, how the disks are being backed, etc.
I'm not familiar with bhyve. I know of it, but not much more.
The iso is the Gentoo minimal install ISO. /path/to/efi.img is the
custom EFI file system created earlier and /dev/gentoop2 is the
partition where the Gentoo install resides.
When the VM boots, /dev/sdb is the gentoo file system and /dev/sda1 is
the EFI partition. Mount the file systems and chroot.
mount /dev/sdb /mnt/gentoo
mount /dev/sda1 /mnt/gentoo/efi
arch-chroot /mnt/gentoo
The arch-chroot is a new one on me. A quick search makes me think it's
just a wrapper around chroot with some creature features. Cool.
In the chroot, backup the old grub config and generate a new one:
cp -r /boot/grub /root
What is your goal of copying /boot/grub to root's home directory; /root?
grub-install --target=x86_64-efi --efi-directory=/efi --removable
grub-mkconfig -o /boot/grub/grub.cfg
I did not need to adjust the fstab since my fstab was empty. Make sure
to check it if you have mounts in there.
fstab is one of those things that's going to be very pointy if you use
device paths. Less so if you use UUID and / or labels.
Exit the chroot and shut down the VM:
exit
shutdown -hP now
The VM can now be booted without the installation medium and boots
just fine:
bhyve -A -H -P -c 2 -m 2G -s 0,hostbridge \
-s 1,lpc -s 2:0,ahci,hd:/path/to/efi.img,hd:/dev/gentoodevicep2 \
-s 4,virtio-net,tap4 -s 29,fbuf,tcp=0.0.0.0:5901,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd testvm
It is doable (since I did it), but how easy it is will depend on your
distribution.
Sure. Lots of things are possible with Linux.
Dr. Ian Malcolm from Jurassic Park comes to mind: ... your scients were
so preoccupied with whether or not they could that they didn't stop to
think if they should.
I'm all for a proof of concept for academic reasons. But I don't think
I'd want to do many of the academic things that I've done in production.
At least not as I did them.
I have not used Slackware so I am afraid I don't really know how easily
the equivalent of what I did for my Gentoo installation would be.
:-)
On Fri, 22 Dec 2023 11:41:41 -0000 (UTC), Andreas Kempe wrote:
Den 2023-12-21 skrev David Chmelik <dchmelik@gmail.com>:
Can one use physical partitions one multi-boots from in Bhyve in
FreeBSD UNIX and keep multi-booting? I'd like to use Slackware
GNU/Linux (and other OS) in Bhyve but still keep it on its partition.
To not just talk out of my ass, I tried this with my Gentoo installation
and got it to work.
Good to see some discussion ending up with people testing this and getting
it working and explaining that for Gentoo GNU/Linux (which I'd also try
this with, without UEFI).
I looked for documentation on FreeBSD.org and BHyVe.org but didn't find
much. Is there a manual somewhere in (e)textbook format other than just
the manual page, or does one have to learn everything from that? I've
found textbooks to be a bit more descriptive than many of the manual pages (some cryptic, some historically removed with the AT&T/whatever fiasco).
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 379 |
Nodes: | 16 (2 / 14) |
Uptime: | 43:49:44 |
Calls: | 8,141 |
Calls today: | 4 |
Files: | 13,085 |
Messages: | 5,857,952 |