Re:Comparison of Rust async and Linux thread context switch ...
by richardwhiuk on Sunday February 14, 2021 @08:06AM
(#26132099)
If you have downstream nodes which may have large amounts of latency in some scenarios, then you may need a huge thread pool.
If you add a huge thread pool, and then those downstreams don't have a large latency, then you end up accepting a huge amount of work and then are CPU starved.
So in order to correctly size your thread pool, you need to understand all your downstream latency, and adapt to it.
Compared to an async runtime, which just handles this scenario, it's very painful.
Even if you get this roughly right, the scheduler is very unhappy when you have lots of threads - it tends to make incorrect scheduling decisions.
If you add a huge thread pool, and then those downstreams don't have a large latency, then you end up accepting a huge amount of work and then are CPU starved.
So in order to correctly size your thread pool, you need to understand all your downstream latency, and adapt to it.
Compared to an async runtime, which just handles this scenario, it's very painful.
Even if you get this roughly right, the scheduler is very unhappy when you have lots of threads - it tends to make incorrect scheduling decisions.