[continued from previous message]

dedicated storage (say NVME) or a ram disk image. It supports a host console running over stdin/stdout. It supports explicit locations such as /dev/ nvme0ns1:/boot/loader/gerbil.conf for where to load filesystems from. It supports ZFS boot environments, including the boot-once feature.

Additional details about kboot, what it supports and some general background can be found in Warner’s BSDcan talk (slides linked above).

FreeBSD/aarch64 now can boot from Linux in a LinuxBoot environment, with support and functionality comparable to loader.efi(8). Memory layout passed in for GICv3 workarounds. Need patch for aarch64 kernel for the GICv3 workaround ( https://reviews.freebsd.org/D40902).

FreeBSD/amd64 support is in progress and is maybe 80% done. The amd64 boot environment places more requirements on the boot loader to provide data for the kernel than aarch64, due to amd64 being an older port. All sources for data in the BIOS environment had to be provided by the boot loader since the kernel had no access to them from long mode. While UEFI and ACPI provide ways for the kernel to get this data, much of the data must still be provided by the boot loader. The kernel panics during initialization since all these prerequisites have not been discovered and implemented.

PowerPC builds, but nothing more of its state is known. Attempts to acquire a suitable Playstation 3 proved to be too time consuming for the author.

Help Needed

1. loader.kboot(8) needs to be written. It should document how to use
loader.kboot, how to create images, and the use cases that work today.

2. Finish amd64 support.

3. The current elf arch-specific metadata code is copied from efi. Unifying
the kboot and efi copies is needed. While they are mostly the same, sharing
is complicated by remaining compile-time differences. In addition, the
build infrastructure makes sharing awkward.

4. It would be nice to add riscv64 support.

5. PowerPC testing (it has been untested since the refactoring started).

6. Creating a script to repackage EDK-II image (say, from QEMU) as a
linux-boot image with a Linux kernel built on FreeBSD for CI testing.

7. Testing it from the coreboot LinuxBoot.

Sponsored by: Netflix, Inc

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Userland

Changes affecting the base system and programs in it.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

OpenSSL 3 in base

Links:
OpenSSL Downloads URL: https://www.openssl.org/source/
OpenSSL 3.0 Has Been Released! URL: https://www.openssl.org/blog/blog/2021/09/ 07/OpenSSL3.Final/
openssl-fipsinstall URL: https://www.openssl.org/docs/man3.0/man1/ openssl-fipsinstall.html

Contact: Pierre Pronchery <pierre@freebsdfoundation.org>

Pierre has been tasked with importing OpenSSL 3 into the base system.

OpenSSL is a library for general-purpose cryptography and secure communication. It provides an open source implementation of the SSL and TLS network protocols, which are widely used in applications such as e-mail, instant messaging, Voice over IP (VoIP), or more prominently the global Web (aka HTTPS). Assuming that the Apache and nginx web servers use OpenSSL, their combined market share for web traffic exceeds 50%, cementing the leadership and critical importance of OpenSSL as part of the infrastructure of the Internet.

Since its initial release in August 2016, the 1.1 branch of OpenSSL has been adopted by most Linux and BSD systems, while remaining supported by the upstream maintainers through an LTS (long term support) policy. However, official support is planned to end in the middle of September this year, and it became urgent and necessary to consider adopting its successor for LTS, the 3.0 branch.

OpenSSL has largely outgrown its ancestor SSLeay, now shipping over half a million single lines of code (SLOC) split in over two thousand files. Perhaps as a consequence, its build system is relatively complex and normally requires Perl, which was removed from base more than twenty years ago for FreeBSD 5.0-RELEASE. Thankfully however, it was possible to import and setup OpenSSL 3.0.9 the FreeBSD way, and it is now part of the base system as planned for FreeBSD 14.0-RELEASE.

To describe OpenSSL 3 as a major release is an understatement. First, its problematic licensing model has finally been solved, with a complete switch to the Apache License 2.0. Then, OpenSSL 3 introduces the concept of provider modules. While obsolete cryptographical algorithms have been isolated to a legacy module, it is also possible to restrict the implementation to the standards part of FIPS with the fips module. The latter can then benefit from a dedicated certification process, and be validated officially (like the 3.0.8 release at the time of writing).

Moreover, the updated library comes with a version bump, as applications using OpenSSL 1.1 need to be recompiled to use 3.0. Many API functions have been deprecated and replaced with newer, more generic alternatives, however it is still possible to explicitly request older APIs and have OpenSSL 3 expose them accordingly. This possibility has been leveraged in FreeBSD to help with the transition, where a number of libraries and applications have simply been configured to request the OpenSSL 1.1 API. These components will be updated progressively over time in order to consume OpenSSL 3’s native API instead.

While there is a known performance impact associated with the update when consuming small input block sizes, it was found to be marginal when working with blocks of 1 KB and above. Another challenge lies with the FIPS provider module, which currently requires some manual steps in order to have it working. We are currently looking for a solution to ship FreeBSD with a functional FIPS provider by default.

Sponsor: The FreeBSD Foundation

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Linux compatibility layer update

Links:
Linuxulator status Wiki page URL: https://wiki.freebsd.org/Linuxulator
Linux app status Wiki page URL: https://wiki.freebsd.org/LinuxApps

Contact: Dmitry Chagin <dchagin@FreeBSD.org>

The goal of this project is to improve FreeBSD’s ability to execute unmodified
linux(4) binaries.

As of cbbac5609115, preserving an fpu xsave state across signal delivery on amd64 is implemented. That makes it possible to run modern golang with preemptive scheduler on.

The new facility to specify an alternate ABI root path was added to the namei (9). Previously, to dynamically reroot lookups, every linux(4) syscall where path names translation is needed required a bit of ugly code and used kern_alternate_path() which does not properly resolve symlinks with leading / in the target. For now a non-native ABI (i.e., linux(4)) uses one call to pwd_altroot() during exec-time into that ABI to specify its root directory (e.g., /compat/ubuntu) and forget about path names translation. That makes possible chroot into the Ubuntu compat without having to fix such symlinks by hand.

In total, over 10 bugs were fixed; glibc-2.37 tests suite reports less than 70 failed tests.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Service Jails — automatic jailing of rc.d services

Links:
D40369: Extend /usr/bin/service with the possibility to set ENV vars URL: https://reviews.freebsd.org/D40369
D40370: Infrastructure for automatic jailing of rc.d-services URL: https:// reviews.freebsd.org/D40370
D40371: automatic service jails: some setup for full functionality of the services in automatic service jails URL: https://reviews.freebsd.org/D40371

Contact: Alexander Leidinger <netchild@FreeBSD.org>

Service jails extend the rc(8) system to allow automatic jailing of rc.d services. A service jail inherits the filesystem of the parent host or jail, but uses all other limits of the jail (process visibility, restricted network access, filesystem mounting permissions, sysvipc, …​) by default. Additional
configuration allows inheritance of the IPs of the parent, sysvipc, memory page locking, and use of the bhyve virtual machine monitor (vmm(4)).

If you want to put e.g. local_unbound into a service jail and allow IPv4 and IPv6 access, simply change rc.conf(5) to have:

local_unbound_svcj_options=net_basic
local_unbound_svcj=YES

While this does not have the same security benefits of a manual jail setup with a separate filesystem and IP/VNET, it is much easier to setup, while providing some of the security benefits of a jail like hiding other processes of the same user.

The patches in the links are a rewrite of what I presented in 2019. The main difference is that an ENV variable is used to do more rational tracking and as such, requires a change to service(8).

My intent is to commit D40369 before the branch of stable/14. I will not commit D40370 or D40371 before 14.0 is released and both will benefit from more eyes.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Security Sandboxing Using ktrace(1)

Links:
ktrace branch URL: https://github.com/jakesfreeland/freebsd-src/tree/ff/ktrace

Contact: Jake Freeland <jfree@FreeBSD.org>

Capsicumization With ktrace(1)

This report introduces an extension to ktrace(1) that logs capability violations for programs that have not been Capsicumized.

The first logical step in Capsicumization is determining where your program is raising capability violations. You could approach this issue by looking through the source and removing Capsicum-incompatible code, but this can be tedious and requires the developer to be familiar with everything that is not allowed in capability mode.

An alternative to finding violations manually is to use ktrace(1). The ktrace (1) utility logs kernel activity for a specified process. Capsicum violations occur inside of the kernel, so ktrace(1) can record and return extra information about your program’s violations with the -t p option.

Programs traditionally need to be put into capability mode before they will report violations. When a restricted system call is entered, it will fail and return with ECAPMODE: Not permitted in capability mode. If the developer is doing error checking, then it is likely that their program will terminate with that error. This behavior made violation tracing inconvenient because ktrace(1) would only report the first capability violation, and then the program would terminate.

Luckily, a new extension to ktrace(1) can record violations when a program is NOT in capability mode. This means that any developer can run capability violation tracing on their program with no modification to see where it is raising violations. Since the program is never actually put into capability mode, it will still acquire resources and execute normally.

Violation Tracing Examples

The cap_violate program, shown below, attempts to raise every type of violation that ktrace(1) can capture:

# ktrace -t p ./cap_violate
# kdump
1603 ktrace CAP system call not allowed: execve
1603 foo CAP system call not allowed: open
1603 foo CAP system call not allowed: open
1603 foo CAP system call not allowed: open
1603 foo CAP system call not allowed: open
1603 foo CAP system call not allowed: readlink
1603 foo CAP system call not allowed: open
1603 foo CAP cpuset_setaffinity: restricted cpuset operation
1603 foo CAP openat: restricted VFS lookup: AT_FDCWD
1603 foo CAP openat: restricted VFS lookup: /
1603 foo CAP system call not allowed: bind
1603 foo CAP sendto: restricted address lookup: struct sockaddr { AF_INET, 0.0.0.0:5000 }
1603 foo CAP socket: protocol not allowed: IPPROTO_ICMP
1603 foo CAP kill: signal delivery not allowed: SIGCONT
1603 foo CAP system call not allowed: chdir
1603 foo CAP system call not allowed: fcntl, cmd: F_KINFO
1603 foo CAP operation requires CAP_WRITE, descriptor holds CAP_READ 1603 foo CAP attempt to increase capabilities from CAP_READ to CAP_READ,CAP_WRITE

The first 7 system call not allowed entries did not explicitly originate from the cap_violate program code. Instead, they were raised by FreeBSD’s C runtime
libraries. This becomes apparent when you trace namei translations alongside capability violations using the -t np option:

# ktrace -t np ./cap_violate
# kdump
1632 ktrace CAP system call not allowed: execve
1632 ktrace NAMI "./cap_violate"
1632 ktrace NAMI "/libexec/ld-elf.so.1"
1632 foo CAP system call not allowed: open
1632 foo NAMI "/etc/libmap.conf"
1632 foo CAP system call not allowed: open
1632 foo NAMI "/usr/local/etc/libmap.d"
1632 foo CAP system call not allowed: open
1632 foo NAMI "/var/run/ld-elf.so.hints"
1632 foo CAP system call not allowed: open
1632 foo NAMI "/lib/libc.so.7"
1632 foo CAP system call not allowed: readlink
1632 foo NAMI "/etc/malloc.conf"
1632 foo CAP system call not allowed: open
1632 foo NAMI "/dev/pvclock"
1632 foo CAP cpuset_setaffinity: restricted cpuset operation
1632 foo NAMI "ktrace.out"
1632 foo CAP openat: restricted VFS lookup: AT_FDCWD
1632 foo NAMI "/"
1632 foo CAP openat: restricted VFS lookup: /
1632 foo CAP system call not allowed: bind
1632 foo CAP sendto: restricted address lookup: struct sockaddr { AF_INET, 0.0.0.0:5000 }
1632 foo CAP socket: protocol not allowed: IPPROTO_ICMP
1632 foo CAP kill: signal delivery not allowed: SIGCONT
1632 foo CAP system call not allowed: chdir
1632 foo NAMI "."
1632 foo CAP system call not allowed: fcntl, cmd: F_KINFO
1632 foo CAP operation requires CAP_WRITE, descriptor holds CAP_READ 1632 foo CAP attempt to increase capabilities from CAP_READ to CAP_READ,CAP_WRITE

In practice, capability mode is always entered following the initialization of the C runtime libraries, so a program would never trigger those first 7 violations. We are only seeing them because ktrace(1) starts recording violations before the program starts.

This demonstration makes it clear that violation tracing is not always perfect. It is a helpful guide for detecting restricted system calls, but may not always parody your program’s actual behavior in capability mode. In capability mode, violations are equivalent to errors; they are an indication to stop execution. Violation tracing is ignoring this suggestion and continuing execution anyway, so invalid violations may be reported.

The next example traces violations from the unzip(1) utility (pre-Capsicumization):

# ktrace -t np unzip foo.zip
Archive: foo.zip
creating: bar/
extracting: bar/bar.txt
creating: baz/
extracting: baz/baz.txt
# kdump
1926 ktrace CAP system call not allowed: execve
1926 ktrace NAMI "/usr/bin/unzip"
1926 ktrace NAMI "/libexec/ld-elf.so.1"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/etc/libmap.conf"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/local/etc/libmap.d"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/var/run/ld-elf.so.hints"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libarchive.so.7"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/lib/libarchive.so.7"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libc.so.7"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libz.so.6"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libbz2.so.4"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/lib/libbz2.so.4"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/liblzma.so.5"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/lib/liblzma.so.5"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libbsdxml.so.4"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libprivatezstd.so.5"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/lib/libprivatezstd.so.5"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libcrypto.so.111"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libmd.so.6"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libthr.so.3"
1926 unzip CAP system call not allowed: readlink
1926 unzip NAMI "/etc/malloc.conf"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/dev/pvclock"
1926 unzip NAMI "foo.zip"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/etc/localtime"
1926 unzip NAMI "bar"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: mkdir
1926 unzip NAMI "bar"
1926 unzip NAMI "bar"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "bar/bar.txt"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "bar/bar.txt"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: mkdir
1926 unzip NAMI "baz"
1926 unzip NAMI "baz"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz/baz.txt"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz/baz.txt"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD

The violation tracing output for unzip(1) is more akin to what a developer would see when tracing their own program for the first time. Most programs link against libraries. In this case, unzip(1) is linking against libarchive(3), which is reflected here:

1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/lib/libarchive.so.7"
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/usr/lib/libarchive.so.7"

The violations for unzip(1) can be found below the C runtime violations:

1926 unzip NAMI "foo.zip"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: open
1926 unzip NAMI "/etc/localtime"
1926 unzip NAMI "bar"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: mkdir
1926 unzip NAMI "bar"
1926 unzip NAMI "bar"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "bar/bar.txt"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "bar/bar.txt"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip CAP system call not allowed: mkdir
1926 unzip NAMI "baz"
1926 unzip NAMI "baz"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz/baz.txt"
1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD
1926 unzip NAMI "baz/baz.txt"
1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD

In this instance, unzip(1) is recreating the file structure contained in the zip archive. Violations are being raised because the AT_FDCWD value cannot be used in capability mode. The bulk of these violations can be fixed by opening AT_FDCWD (the current directory) before entering capability mode and passing that descriptor into openat(2), fstatat(2), and mkdirat(2) as a relative reference.

Violation tracing may not automatically Capsicumize programs, but it is another tool in the developer’s toolbox. It only takes a few seconds to run a program under ktrace(1) and the result is almost always a decent starting point for sandboxing your program using Capsicum.

Sponsor: FreeBSD Foundation

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

NVMe over Fabrics

Links:
nvmf2 branch URL: https://github.com/bsdjhb/freebsd/tree/nvmf2

Contact: John Baldwin <jhb@FreeBSD.org>

NVMe over Fabrics enables communication with a storage device using the NVMe protocol over a network fabric. This is similar to using iSCSI to export a storage device over a network using SCSI commands.

NVMe over Fabrics currently defines network transports for Fibre Channel, RDMA, and TCP.

The work in the nvmf2 branch includes a userland library (lib/libnvmf) which contains an abstraction for transports and an implementation of a TCP transport. It also includes changes to nvmecontrol(8) to add 'discover', 'connect', and 'disconnect' commands to manage connections to a remote controller.

The branch also contains an in-kernel Fabrics implementation. nvmf_transport.ko contains a transport abstraction that sits in between the nvmf host (initiator in SCSI terms) and the individual transports. nvmf_tcp.ko contains an implementation of the TCP transport layer. nvmf.ko contains an NVMe over Fabrics host (initiator) which connects to a remote controller and exports remote namespaces as disk devices. Similar to the nvme(4) driver for NVMe over PCI-express, namespaces are exported via /dev/nvmeXnsY devices which only support simple operations, but are also exported as ndaX disk devices via CAM. Unlike nvme(4), nvmf(4) does not support the nvd(4) disk driver. nvmecontrol(8) can be used with remote namespaces and remote controllers, for example to fetch log pages, display identify info, etc.

Note that nvmf(4) is currently a bit simple and some error cases are still a TODO. If an error occurs, the queues (and backing network connections) are dropped, but the devices stay around, with I/O requests paused. nvmecontrol reconnect can be used to connect a new set of network connections to resume operation. Unlike iSCSI which uses a persistent daemon (iscsid(8)) to reconnect after an error, reconnections must be made manually.

The current code is very new and likely not robust. It is certainly not ready for production use. Experienced users who do not mind all their data vanishing in a puff of smoke after a kernel panic, and who have an interest in NVMe over Fabrics, can start testing it at their own risk.

The next main task is to implement a Fabrics controller (target in SCSI language). Probably a simple one in userland first followed by a "real" one that offloads the data handling to the kernel but is somewhat integrated with ctld(8) so that individual disk devices can be exported via iSCSI or NVMe, or via both using a single config file and daemon to manage all of that. This may require a fair bit of refactoring in ctld to make it less iSCSI-specific. Working on the controller side will also validate some of the currently under-tested API design decisions in the transport-independent layer. I think it probably does not make sense to merge any of the NVMe over Fabrics changes into the tree until after this step.

Sponsored by: Chelsio Communications

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Kernel

Updates to kernel subsystems/features, driver support, filesystems, and more.

Boot Performance Improvements

Links:
Wiki page URL: https://wiki.freebsd.org/BootTime
BSDCan talk slides URL: https://www.bsdcan.org/events/bsdcan_2023/sessions/ session/116/slides/44/BSDCan23-Firecracker.pdf

Contact: Colin Percival <cperciva@FreeBSD.org>

Colin is coordinating efforts to speed up the FreeBSD boot process.

Recent efforts have moved from EC2 to the Firecracker virtual machine manager, which provides a very minimalist environment; stripping the boot process down to the bare minimum makes it easier to identify the remaining time and determine whether it can be optimized further.

With some experimental patches to both FreeBSD and Firecracker, it is now possible to boot a FreeBSD kernel in under 20 ms.

Some of the recent improvements were discussed in Colin’s Porting FreeBSD to Firecracker session at BSDCan.

This work is supported by his FreeBSD/EC2 Patreon.

Sponsor: https://www.patreon.com/cperciva

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

CI Test Harness For Bootloader

Links:
FreeBSD Wiki GSoC Page
Github Project Link

Contact: Sudhanshu Mohan Kashyap <smk@FreeBSD.org>

FreeBSD supports multiple architectures, file systems, and disk-partitioning schemes. I am trying to write a Lua script which would allow for testing boot loader of all the architecture combinations supported in the first and second-tier support, and provide a report on any broken combinations and expected functionality. If time permits, further exploration could be done to integrate the script into the existing build infrastructure (either Jenkins or Github Actions) to generate a comprehensive summary of the test results.

Currently any changes made by developer might inhibit the ability of the operating system to boot in some specific environment. These scripts provide assurance that changes do not cause regressions for the tested environments. The scripts are designed to be efficient and much less expensive than a full make universe required today. These attributes allow developers to routinely use the script, and allow integration into the CI pipelines without undue cost.

Currently script related work seems to be on track, but certainly ahead I will need to find all different kinds of QEMU recipes to test different environments. If anyone has any kind of working QEMU recipe for currently released versions of FreeBSD, feel free to send to me via mail at smk@FreeBSD.org .

Sponsor: The Google Summer of Code '23 program

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Physical memory compaction for the FreeBSD kernel

Links:
GSoC project wiki page URL: https://wiki.freebsd.org/SummerOfCode2023Projects/ PhysicalMemoryAntiFragmentationMechanisms
Differential revision 40575 URL: https://reviews.freebsd.org/D40575 Differential revision 40772 URL: https://reviews.freebsd.org/D40772

Contact: Bojan Novković <bnovkov@FreeBSD.org>

Most modern CPU architectures offer performance boosts by supporting pages that are larger than the standard page size. Unfortunately, allocating such pages can fail due to a high degree of physical memory fragmentation. This work implements physical memory compaction as a means of actively reducing fragmentation in running systems. This work is part of an ongoing Google Summer of Code project, the goal of which is to add various physical memory anti-fragmentation measures to the virtual memory subsystem.

Differential D40575 implements a well-known metric used for quantifying the degree of physical memory fragmentation. Differential D40772 implements physical memory compaction and adds a daemon that monitors the system and performs compaction when needed.

Planned future work includes designing an appropriate benchmarking suite, running tests, and tweaking the code using feedback from reviews and test results. This is still a work in progress, so any testing, reviews, and feedback would be greatly appreciated.

Sponsor: The Google Summer of Code '23 program

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Increasing MAXCPU

Links:
Review D36838: amd64: Bump MAXCPU to 1024 (from 256) URL: https:// reviews.freebsd.org/D36838

Contact: Ed Maste <emaste@FreeBSD.org>

The default amd64 and arm64 FreeBSD kernel configurations currently support a maximum of 256 CPUs. A custom kernel can be built with support for larger core counts by setting the MAXCPU kernel option. However, commodity systems with more than 256 CPUs are becoming available and will be increasingly common during FreeBSD 14’s support lifecycle. We want to increase the default maximum
CPU count to 1024 to support these systems "out of the box" on FreeBSD 14.

A number of changes have been made to support a larger default MAXCPU, including fixing the userland maximum for cpuset_t at 1024. Changes have also been made to avoid static MAXCPU-sized arrays, replacing them with on-demand memory allocation.

Additional work is required to continue reducing static allocations sized by MAXCPU and addressing scalability bottlenecks on very high core count systems, but the goal is to release FreeBSD 14 with a stable ABI and KBI with support for large CPU counts.

Sponsor: The FreeBSD Foundation

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

SquashFS port for FreeBSD kernel

Links:
Wiki page URL: https://wiki.freebsd.org/SummerOfCode2023Projects/ PortSquashFuseToTheFreeBSDKernel
Source code URL: https://github.com/Mashijams/freebsd-src/tree/gsoc/squashfs

Contact: Raghav Sharma <raghav@FreeBSD.org>

SquashFS is a read-only file system that lets you compress whole file systems or single directories very efficiently. Support for it has been built into the Linux kernel since 2009 and is very common in embedded Linux distributions. The goal of this project is to add SquashFS support for the FreeBSD kernel, with the aim of being able to boot FreeBSD from an in-memory SquashFS file system.

Currently, the driver is compatible with the 13.2 FreeBSD release. The driver is able to correctly parse the SquashFS disk file with working mount(8) support. Linux SquashFS supports many compression options like zstd, lzo2, zlib, etc…​ based on the user’s preference, and of course, our driver supports
all those compressions as well.

Planned future work includes adding directories, files, extended attributes, and symlinks read support. The project is still a work in progress under the mentorship from Chuck Tuffli and is expected to complete by the end of the Google Summer of Code program.

Sponsor: The Google Summer of Code 2023 program

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Pf Improvements


[continued in next message]

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)