"Plain" OS threads certainly have their place and the OS does a fine job scheduling them. It's just that the more information you have about the threads' behavior the better they can be scheduled (you can reduce latencies by keeping related fibers on the same core to share cache; that's one of the things Quasar does).

So the increased latency of the OS scheduler has little to do with the number of layers between your application and the hardware, and a lot to do about assumptions the OS can make about your code.

You most certainly want a general-purpose OS scheduler, it's just that many applications can benefit from user-level lightweight threads.

For me, when it comes to really drastic solutions like running bare metal or building a custom OS then if you have to ask if you should then you shouldn't. But to echo the others, it depends on what you are doing.

I hope I don't come off as a creeper because I was very curious on what you are working on and dug through your comments. Is it this BareMetalOS project?

You could also be asking about whether it is time to ditch "commodity" hardware (which by the way is not massively parallel).

The answer is, of course, it depends on what your priorities are.

I've seen projects that write OS's from the ground up for a wide variety of reasons (security, correctness, optimization of use case, etc.) But very rarely is it for price reasons.

I wrote a comment about a paper I read that ditched tcp and had the application handle it to get some performance gains earlier, so you might want to page up to see it if you haven't already (Network Specialization for Performance).

How is it that you intend to do an OS startup? That's a pretty tough nut to crack.

you might want to check out the rustboot project then. one of the use-cases is to have rust boot on a hypervisor that then just has not much more than the runtime like you're after