Everyone else is guessing how may "fork" commands to throw out the OS. The OS itself has much more information at its disposal about what resources are being used, where the bottle necks are etc.

An elegant design on doing this in userspace probably boils down to going well on the awy to making a micro kernel.

Specifically one failure mode I've seen with Ninja is that it starts 12 processes for my 12-threaded CPU, each process eats 2 gigabytes of RAM, and I only have 16 gigabytes of RAM total, so it sends my system into swap or oomkiller. Ideally when the RAM fills up (whether because of compilers spawned by Ninja, or because I opened Discord), Ninja would kill one of the build processes and send it back onto the build queue, and reduce the number of compilers running at a time by 1 until I either tell it, or (some heuristic like) over 1.5 times the maximum RAM taken by a single compiler so far gets freed up.

I don't know if this is best handled by the OS, userspace Ninja, or some combination (like cgroups).

This sounds a lot like the idea behind Dispatch[0] (aka, Grand Central Dispatch) on Apple systems.

[0] https://developer.apple.com/documentation/DISPATCH