The Erlang runtime uses non-blocking IO calls to the kernel but exposes blocking functions on top of this. Is such a function really blocking? It depends on what level you're looking at; it's blocking at Erlang level but non-blocking at kernel level. I think this is the source of the confusion. You are of course correct in saying "If you have a blocking call you still have to block an OS thread somewhere." if you're looking at kernel level, but not Erlang level (since it can translate blocking to non-blocking).
> If you have non-blocking IO then it seems like you could easily implement an M:N threading model as a library without making it awkward to interact with.
Yes, but your language/system has to have some mechanism to manage control flow. Examples: F# does what you ask for via asynchronous workflows. Ruby via fibers/EM-Synchrony. Java via a bytecode weaver like Kilim.
If this can't be done, then that brings me back to my previous point. It seems like the important thing is that you have non-blocking IO and not that you have an M:N threading model in the runtime. If you have non-blocking IO then it seems like you could easily implement an M:N threading model as a library without making it awkward to interact with.
Is there some assumption I'm making that doesn't make sense?