> In which ways is OpenBSD more in control of userspace applications, and in which ways is Linux more loosely coupled?

One of the major differences between Unix and Linux is which parts of the system are in the source tree.

Linux is technically just the kernel, and a Linux distribution will put together the base libraries and userland programs and kernel along with various tools for installing and managing.

Kernel space programs are generally the kernel itself and device drivers.

Base userland programs are things like `cat` and `ls` and more that allow for manipulating and inspecting parts of the system. User-space programs are any userland program which is not in kernel-space, including third party packages and custom software written by the user.

Userland programs need to be linked against libraries, which is where libc comes in.

In most BSD Unix systems, the source tree will include the kernel and the base userland programs together.

That means that OpenBSD's versions of `libc` and `ls` are maintained in the same source tree as the kernel, allowing for much tighter coupling and integration for changes.

By contrast, the Linux distributions are generally using some versions of GNU C and GNU Coreutils, perhaps with patches downstream.

So a core change to the Linux kernel must make its way upstream to the user space application and then downstream from that, the distribution must integrate both.

This is much more loosely coupled and should hopefully illustrate the difficulty in coordinating such a change in Linux.

The coupling here is specifically the lockstep of the kernel syscall ABI and libc.

It's just ABI instability. In BSD, Solaris, and MacOS the kernel and libc are distributed by essentially the same people as an atomic unit, and the syscall layer changes over time. On Linux, it's the syscall layer that's stable.

It's entirely doable in Linux too. Introduce new syscalls for the new interface, plus one lockdown functionality that disables the old interface so sysadmins/distributions can switch to the new interface. Of course this can and should be coupled with compile time switching, so eventually the old interface will become deprecated, and if no one steps up to maintain it, it will be removed.

It's simply a different approach to doing backward compatibility. (Instead of maintaining very different trees for different versions, the functionality is concurrently available in multiple versions, but with feature switches [compile- and run-time configuration].)

> and if no one steps up to maintain it, it will be removed.

"You don't break userspace". How many decades until people who no longer have source for the broken binaries that they run will stop complaining?

How long until sysadmins know to turn the knobs on?

How many knobs are sustainable? How much will they interact, and in how many ways will those interactions expose vulnerabilities?

The Linux way to introduce this is hell for everyone involved.

If no one uses it, removing it does not break userspace. Removal of features happen regularly.

How many? Dunno, doesn't matter. If there are users they are usually willing to step up to maintain it, and that makes having parallel APIs not a problem (the old one becomes a wrapper).

Furthermore UNIX/BSD faces the same problem with every other program that lives out of tree. (Which is the majority of them anyway.)

Knobology is always an endless debate, yet also regularly done without much fuss, usually simply by what the maintainer(s) decide to provide.

As done on the Trebelified Linux in Android.

Standard Linux drivers are considered legacy, with all new drivers using their own processes talking via Android IPC.

Userspace is all about Android Java, ISO C, ISO C++, and NDK APIs, everything else is considered off hands and there is also some gatekeeping via LinuxSE, securecomp of whatever else is allowed.

Right, but the abstraction later for Treble is the HAL boundary and not the syscall boundary. The seccomp filter is unrelated and IIRC relatively permissive.

Syscalls are not part of the NDK stable APIs contract, so although seccomp isn't as extensive as it might be, there are zero guarantees about blocking further syscalls.

Nothing you need to write a driver is part of the NDK stable API contract either. They're orthogonal concepts.

Sure, however we are speaking about general purpose access to syscalls from user space here.

Only Android OEMs get to publish drivers.

> Did Solaris and macOS also do that for security reasons? (A linked article mentions ABI instability as the reason, but maybe there's more to it.)

That's all there is to it (I did the Solaris port).

> In which ways is OpenBSD more in control of userspace applications

They port and package them themselves.