Re:Memory efficiency of parallel IO operations in Python
by blattimwind on Tuesday March 13, 2018 @09:12AM
(#16575800)
> Can somebody explain to me why async IO is so important and why it is better than using the operating system scheduler?
You still do.
The difference is with thread-based IO you block a task until the IO scheduler is done with one operation, while with async IO you block a task until the IO scheduler is done with any operation.
The reason why async can be more efficient is that you have fewer tasks and possibly less task-housekeeping-overhead.
It's important to note that disk I/O generally has poor support for asynchronous operation (because disk I/O traditionally meant that high I/O concurrency brutally murders throughput, which has changed). It's something that's being worked on for Linux, though.
You still do.
The difference is with thread-based IO you block a task until the IO scheduler is done with one operation, while with async IO you block a task until the IO scheduler is done with any operation.
The reason why async can be more efficient is that you have fewer tasks and possibly less task-housekeeping-overhead.
It's important to note that disk I/O generally has poor support for asynchronous operation (because disk I/O traditionally meant that high I/O concurrency brutally murders throughput, which has changed). It's something that's being worked on for Linux, though.