Re:Comparison of Rust async and Linux thread context switch ...
by PixelOfDeath on Friday February 12, 2021 @12:05PM
(#26115795)
I hat very good experience with using buffer variables to copy-"prefetch" unpredictable costly fetches. E.g. from cache lines that get touched by several cores for communication.
And I only actual use them after one iteration of whatever I do. So the core could fetch the memory content without actually having to stall because I do not use it until later.
I'm not sure how realistic that is inside a kernel thread scheduler, but it sure is useful in user space for task based libraries.
Low context switch latency only matters when the number of tasks is very small (their data all fits in the cache), and the workload is entirely computational. Otherwise, even the fastest implementation is ~60 ns, which is the cost of a cache-miss, and the compiler can't optimise things into a simple goto because the dispatch goes through a scheduler that has a megamorphic call-site.
So memory is much more important for I/O use-case throughput, and while it is true that the kernel doesn't commit the full stack memory on thread creation, it's misleading to think that you get good memory usage. For one, once the memory is committed, it's never uncommitted (although it can be paged out). For another, the granularity is that of a page, i.e. at least 4K, which can often be much higher than what a task requires.
> It is hard to pin down exactly how the alleged advantages would arise.
For I/O use-cases the answer is here:
> the async version uses about 1/20th as much memory as the threaded version.
This could translate to 20x throughput -- due to Little's law -- although usually less because there are other limits, like network saturation.