As someone who's done a bit of game development in the past I can easily say that you'll fall flat on your face in some fields if you don't write your own protocols. There are simply no protocols that operate well in every use case and there are also use-cases without standardized protocols.
For extremely high performance operations you NEED to use a custom protocol in some case to squeeze every bit of power out of your applications.
From a game development perspective you need to handle two types of content: content that isn't really ordered by time occurrence and content that needs to happen sequentially.
Because of this you can correctly optimize your networking stack for those needs and get your networking system really fast.
Some people have taken to using TCP with a JSON message passing system and that just doesn't work large scale. You'll have huge latency issues and with a big enough game you have to do some serious bit-packing to make it cost effective to host servers for.
Now I'm speaking generally and this is just from my experience but there is definitely still a reason to write your own network code.
You use green threads instead of native threads because native threads have space overhead, not because they have time overhead. Attempting to spawn 100k OS threads will do strange things to most kernel scheduling algorithms; they're not optimized for that use-case.
The main overhead of a thread, green or native, is the stack. The size of the stack is independent of whether you use native or green threads. Go's small stacks are actually made possible by its GC, not its choice of 1:1 or M:N. In musl, for example, you can have 2KB stacks [1] with 1:1.
I haven't seen a benchmark of huge numbers of native threads vs. a userland scheduler, but I have a hard time imagining that a userland scheduler will beat the kernel's scheduler. The kernel scheduler has a much more global picture of the system compared to userland.
[1]: https://github.com/rofl0r/musl/blob/d05aaedaabd4f5472c233dbb...
In most of the comparisons I've seen (usually for Erlang), worst-case latency was the important factor, so interaction with the kernel scheduler was avoided as much as possible. In the Erlang runtime, you can pass a switch to cause the userland scheduler-threads to each get bound to a particular processor core, and to cause the kernel scheduler to avoid scheduling anything on those cores. Effectively, this partitions the processor into a set of cores the OS scheduler entirely manages, and a set of cores that the userland scheduler entirely manages.
It's a lot like a machine running a hypervisor with a single VM on it set to consume 100% of available resources—but actually slightly more efficient than that, since a guest can intelligently pin allocate cores to itself and pin its scheduler-threads to them and then just stop thinking about the pinning, while a hypervisor-host is stuck constantly thinking about whether its vCPUs-to-pCPU mapping is currently optimal, with what's basically a black box consuming those vCPUs.
I know with cgroups in Linux you can pin processes to cores pretty easily. Just curious how Erlang makes this easier.
If you mean that in terms of "does the Erlang runtime intelligently take advantage of the fact that its schedulers are pinned to cores to do things you don't get from plain OS-level pinning", I'm not sure.
I think it might, though. This is my impression from reading, a year or so back, the same docs I just linked; you can read them for yourself and form your own opinion if you think this sounds crazy:
It seems like ERTS (the Erlang runtime: BEAM VM + associated processes like epmd and heart) has a pool of "async IO" threads, separate from the regular scheduler threads, that just get blocking syscalls scheduled onto them. Erlang will, if-and-only-if it knows it has pinned schedulers, attempt to "pair" async IO threads with scheduler threads, so that Erlang processes that cause syscalls schedule those syscalls onto "their" async IO threads, and the completion events can go directly back to the scheduler-thread that should contain the Erlang process that wants to unblock in response to them.†
In the default case, if you don't tell ERTS any different, it'll assume you've got one (UMA) CPU with N cores, and will try to pin async IO threads to the same cores as their paired scheduler-threads. This has context-switching overhead, but not much, since 1. the async IO thread is mostly doing kernel select() polling and racing to sleep, and 2. the two threads are in a producer-consumer relationship, like a Unix pipeline, where both can progress independently without needing to synchronize.
If you want, though, you can further optimize by feeding ERTS a CPU map, describing how the cores in your machine are grouped into CPU packages, and how the CPU packages are further grouped into NUMA memory-access groups. ERTS will then attempt to schedule its async IO threads onto a separate core of the same CPU package, or if not possible, the same NUMA group* as the scheduler-thread, to decrease IPC memory-barrier flushing overhead. (The IPC message is still forced to dump from a given core's cache-lines into the CPU or to NUMA local memory, but it doesn't have to go all the way to main memory.)
ERTS will also, when fed a CPU map, penalize the choice in its scheduling algorithm to move an Erlang process to a different CPU package or NUMA group. (It will still do it, but only if it has no other choice.)
---
† This is in contrast to a runtime without "native" green-threads, like the JVM, where even if you've got an async IO pool, it just sees an opaque pool of runtime threads and sends its completion events to one at random, and then it's the job of a framework like Quasar to take time out of the job of each of its JVM runtime threads to catch those messages and route them to a scheduler running on one of said runtime threads.
The same is true of an HVM hypervisor: without both OS support (paravirtualization) plus core pinning inside each VM, a hardware interrupt will just "arrive at" the same pCPU that asked to be interrupted, even if the vCPU that was scheduled on that pCPU when it made the hypercall is now somewhere else. This is why SR-IOV is so important: it effectively gives VMs their own named channels for hardware to address messages to, so they don't get delayed by misdelivery.
For M:N, there are N kernel stacks.
I don't know of any benchmark comparing Go vs. a 1:1 implementation with 2K pthread stack sizes, but I would be surprised if the performance difference is large at all.
And getting even cheaper than that, with the effort to make kernel stacks use virtual memory. As I understand it, once kernel stacks use virtual memory, they'll start out at a single 4k page.
Doesn't using kernel threads imply lots of context switches? Doesn't that tend to be expensive in terms of time on modern architectures?
No, it's actually fewer context switches. That's because each I/O completion event goes straight from the kernel to the code that was waiting on it (1 context switch), not from the kernel to the userland dispatcher to the code that was waiting on it (2 context switches).
If you're willing to put in a lot of compiler work userland thread switching is a function call, literally. I don't 1:1 threads with small stack are going to compete with that.
Yeah, this doesn't apply for context-switches that happen because of plain old pre-emption, but it does happen if the context-switch is because e.g. a network packet wants to arrive at another process in your VM than the one that's currently running.
I would imagine that there's a reason bare-metal IaaS providers have a business model. :)
Switches occur on timeouts (which involve a round trip through the kernel), on I/O events (which also involve a round trip through the kernel), or on goroutine message passing. So in every case except goroutine message sends, the switches don't actually save you a trip through the kernel.
But ok, for reading you might often have a buffered state of readable while in reality it's not and you only get this information through EAGAIN on read, so you won't avoid the syscall there. For writing you most likely always would.
If you're in that world, you're using C, C++, or That if you're feeling dangerous.
Of course it isn't the OpenJDK, rather specialized JVMs like Azul or they code using C style coding with the GC out of the out paths, having profiled the applications with tools like Java Mission Control.
These are the customers driving Oracles effort for value types, AOT compilation and JNI replacement.
Why not just use C and C++, one might ask?
In spite of all these tricks and required knowledge, salaries are lower than for C and C++ developers and overall project costs are anyway lower due to shorter development times for the other tools not available in C and C++.
Hence why Fintech nowadays is looking into languages like Pony, because they don't want to wait for Java 10, if possible.
So an area where Rust might eventually earn some friends.
Support for kernel bypass networking libraries like ibverbs, DPDK (has an old unmaintained Rust wrapper) and other IO kernel bypass libraries such as SPDK and IOAT.
If Rust supported these libraries, I'd much prefer the future based Rust code to a massive event loop in C.
You don't gain as much performance. On Linux, you don't actually gain that much if anything over 1:1 threading. Most of the benefits of goroutines actually comes from the small stacks, which don't have anything to do with M:N and 1:1 to begin with—they're a feature of GC.
As the blog post states, our end goal is to achieve the ergonomics of Go-style M:N without sacrificing nginx levels of performance. With this futures library, we've established the foundations. It would make no sense to give up before even trying.
> And if I need anything better performing, I can still switch to C for specific use cases.
Why would you write networking code in C in 2016, when there are better alternatives available (like this one)?
Rust's philosophy is to have both performance and ergonomics. It rejects the idea that optimal performance requires a cumbersome programming model. I'm not about to give up on that.