Async Python is not faster

Re:Async Python is not faster by wongarsu on Friday June 12, 2020 @09:41AM (#23499116)

Async IO was in large part a response to "how can my webserver handle xx thousand connections per second" (or in the case of Erlang "how do you handle millions of phone calls at once"). Starting 15 threads to do IO works great, but once you wait for hundreds of things at once the overhead from context switching becomes a problem, and at some point the OS scheduler itself becomes a problem

[ Parent | on HN ]

Re:Async Python is not faster by tijsvd on Friday June 12, 2020 @11:12AM (#23500049)

Not really. At least on Linux, the scheduler is O(1). There is no difference between one process waiting for 10k connections, or 10k processes waiting for 1 each. And there is hardly a context switch either, if all these 10k processes use the same memory map (as threads do).

I've tested this extensively on Linux. There is no more CPU used for threads vs epoll.

On the other hand, if you don't get the epoll imementation exactly right, you may end up with many spurious calls. E.g. simply reading slow data from a socket in golang on Linux incurs considerable overhead: a first read that is short, another read that returns EWOULDBLOCK, and then a syscall to re-arm the epoll. With OS threads, that is just a single call, where the next call blocks and eventually returns new data.

Edit: one thing I haven't considered when testing is garbage collection. I'm absolutely convinced that up to 10k connections, threads or async doesn't matter, in C or Rust. But it may be much harder to do GC over 10k stacks than over 8.

[ Parent | on HN ]

Re:Async Python is not faster by dathinab on Friday June 12, 2020 @04:38PM (#23503666)

I recently have read a block with benchmarks doing that for well written C in their use case async io only becomes faster then using threads from around 10k parallel connections. (Through the difference was negligible).

This seems to also be a major behind io_uring.

[ Parent | on HN ]

Re:Async Python is not faster by staticassertion on Friday June 12, 2020 @09:47AM (#23499179)

I don't think this is true? At least, I've never seen the issue of OS threads be that context switching is slow.

The issue is memory usage, which OS threads take a lot of.

Would userland scheduling be more CPU efficient? Sure, probably in many cases. But I don't think that's the problem with handling many thousands of concurrent requests today.

[ Parent | on HN ]