4 months ago we had FreeBSD booting in 25ms. That gives some insight where the bottlenecks might be: https://news.ycombinator.com/item?id=37319180

In general, lots of hardware requires long waits to initialize. As in, program a value then you must wait 50 ms before querying the registers or the hardware will give you non-deterministic garbage with no error indication.

So, it depends on the hardware, though you are unlikely to need more than 1 second on anything other than truly degenerate hardware assuming you are initializing devices in parallel.

But it’s only for VMs. So what hardware are you initializing?

I pass through a GPU and USB hub to a VM running on a machine in the garage. An optical video cable and network compatible USB extender brings the interface to a different room making it my primary “desktop” computer (and an outdated laptop as a backup device). Doesn’t get more silent and cool than this. Another VM on the garage machine gets a bunch of hard drives passed through to it.

That said, hardware passthrough/VFIO is likely out of the current realistic scope for this project. VM boot times can be optimized if you never look for hardware to initialize in the first place. Though they are still likely initializing a network interface of some sort.

“MicroVM” seems to be a term used when as much as possible is stripped from a VM, such as with https://github.com/firecracker-microvm/firecracker

Because of the debug mode

Yes! And because of sandbagging :)

Many years ago I contributed to IncludeOS and that thing could boot a vm in a millisecond under the right circumstances, allowing for things like per request vm isolation. So I suspects it's a mix of hardware circumstances and the way the microkernel is implemented.

Timer calibration is a big time suck.