News for hackers. Stuff that matters.
Re:Libaco: A fast and lightweight C asymmetric coroutine lib... by jandrewrogers on Saturday July 14, 2018 @10:49AM (#17530539)
I use coroutines in C++ frequently as well as threads; they solve different problems. There are two major use cases for coroutines in my experience: they are a much faster concurrency primitive in some cases and they make the design of some complex concurrency problems much simpler. Using coroutines, which are functional in nature, seem complicated until you grok them and at which point the design of such code becomes clear and obvious (like functional programming generally).

The large performance advantage of coroutines comes from two main areas. First, context-switching in coroutines is at least a thousand times faster than with threads. On modern processors for software with fine-grained locking requirements, it is common to see multithreaded applications spending more CPU time on context-switching than actually running application code. Database engines were early adopters of the coroutine model because they have this problem in a major way. Second, you have complete control of the execution schedule ensuring that your code is always doing the best possible thing at the best possible time; schedule-based optimization offers a large throughput improvement by allowing you to dynamically select the task that is most optimal to run in the current context rather than selecting randomly. For a simple example from the database world, most atomic units of work are applying a scan operators to a data page, which are generally very fast -- access latency is a major fraction of the total cost. If you have a thousand queries running concurrently, many of them will touch the same pages at different times. In a coroutine model, when one coroutine pulls a data page into L2 cache, it is inexpensive to immediately schedule the other queries that will need to eventually access that data page to run their operator on the same page at the same time, switching between them sequentially. In multithreaded environments, you tend to end up churning the CPU cache for almost every single page operation. The overhead of most schedule-based optimization in threaded environments is prohibitive, so it largely isn't done.

The simplicity advantage for complex concurrency problems is that you don't need locks, which is the bane of multithreaded programming, with the caveat that all your operations need to be non-blocking. The full state of the various tasks is available to you so conflicts can be eliminated by scheduling the coroutine execution so that they never occur. For all practical purposes, code correctness is evaluated in a single-threaded context. You can design multithreaded code primitives that can inspect their own lock graphs to ensure that deadlocks and other conflicts can always be resolved automatically, database engines tend to use these structures pervasively, but they are both complex and computationally expensive. The disadvantage of coroutines here is that all of your I/O interactions must be non-blocking. Most I/O these days should be non-blocking as a matter of performant design but if you are using coroutines then using blocking I/O is not a sensible option for the developer.

Architecturally, coroutines lead to a model where virtually all data is private to a thread whereas in a multithreaded model most data tends to be shared between threads. There are many concurrency and complexity advantages to making data non-shared but it comes with one major downside -- you need to figure out how to balance compute load in such a model, which is perfectly solvable but not trivial like the multithreaded case. Distributed systems have the same load balancing problem (though few attempt to solve it in a robust way).

Re:Libaco: A fast and lightweight C asymmetric coroutine lib... by ur-whale on Saturday July 14, 2018 @11:54AM (#17530808)
> by allowing you to dynamically select the task that is most optimal to run in the current context rather than selecting randomly

thanks for the very complete explanation, and this sentence is the crux afaic.

[edit]: what do you use to do coroutines in C++?

Re:Libaco: A fast and lightweight C asymmetric coroutine lib... by otabdeveloper2 on Saturday July 14, 2018 @02:54PM (#17531757)
> First, context-switching in coroutines is at least a thousand times faster than with threads.

You misunderstand how context switching works. It doesn't matter how many threads you have -- 5 or 5000. Your OS will switch contexts at a constant rate.

What you perceive as 'context switch overhead' is really process scheduling being too smart for its own good.

99% of the time the problem is solved with a simple round-robin process scheduler. (Called a 'realtime' scheduler in Linux.)

> it is common to see multithreaded applications spending more CPU time on context-switching than actually running application code

No. Like I said, context switching is constant-time overhead for your OS.