It'd be nice if some pthread implementation gave us what we need, but I don't think they do. They probably are too heavy an abstraction, even if implemented in user space, providing too strong guarantees for context switching and scheduling. For example, we're never going to set individual signal handlers on different coroutines, but pthreads guarantee this capability. So even user-level pthreads which don't support preemption will require a system call on context switches, just as ucontext.h's swapcontext does for similar reasons. But as Slava points out, the performance penalty is something that we need to verify in the future with benchmarks.
What would be really nice is if the OS, or at least a standard application framework usable from C++, supported this. But it doesn't, so we build our own.
Yep, I think so too. I don't know of anything that does that on linux, sadly.
Man, I gotta dust off that freebsd partition again.
Just spin up a FreeBSD instance in EC2. :-)
(And while you have it running, please run your favourite stress tests and try to find bugs. I think it's starting to get decently stable now, but it needs lots more testing.)
I said "userland threads" to distinguish it from kernel threads, which are (as they noticed) rather expensive.
What RethinkDB really want, and seem to be headed towards, is a proper M:N threading model: Launch one kernel thread per CPU (N), then spread M userland threads between those.