News for hackers. Stuff that matters.
Re:Efficient IO with io_uring [pdf] by waynesonfire on Monday May 06, 2019 @06:54PM (#19844760)
Can anyone comment on how this interface compares to state-of-the-art async i/o in other *unix o/s? Is Linux playing catch up here or innovating?
Re:Efficient IO with io_uring [pdf] by asveikau on Monday May 06, 2019 @07:54PM (#19845105)
I am not at up to date on this either, but as far as I know, the old status quo was this:

For non-blocking sockets and the like you have what grew out of the readiness model from select and poll. Today on Linux that's epoll.

This model never worked for files on disk. There is the POSIX aio API for that. On Linux, glibc implements that and it's always been based on synchronous I/O in thread pools because the kernel support isn't good. And I guess there's been Linux specific things like io_submit(2), which as the article states only worked with O_DIRECT and sometimes fell back to synchronous I/O.

It says the new mechanism works with both files and sockets, but I imagine it's still reasonable to use epoll for sockets, and only use this new stuff for files on disk. One good thing about the readiness model for sockets is you don't need to allocate memory upfront every potential read the remote host might trigger.

Re:Efficient IO with io_uring [pdf] by Klathmon on Monday May 06, 2019 @11:18PM (#19846170)
This explanation gave me that feeling of how utterly incredible it is at any software works as well as it does.

The amount of work that goes into supporting, debugging, developing, improving, and designing the sheer number of ways that we do things, at literally hundreds of layers of abstractions all which are slightly wrong and each of which does things their own way with their own tradeoffs.

I'm damn glad there are people a lot smarter and more dedicated than I am that keep this stuff running so that I can store a value in localstorage in a browser and have it punch through all of those layers in a fraction of a moment down to writing bits on one of like a dozen storage mediums on any number of OSs via multiple interfaces in multiple browsers.

Re:Efficient IO with io_uring [pdf] by aarongolliver on Tuesday May 07, 2019 @12:06AM (#19846464)
Can you (or someone) explain _why_ the model translate to files on disk? Is it that there's a different set of operations that need to be supported? (you can't seek a socket I suppose)
Re:Efficient IO with io_uring [pdf] by asveikau on Tuesday May 07, 2019 @12:19AM (#19846530)
The model for aio is "here is a buffer, let me know when you've done the operation". Your request can sit in a queue somewhere or whatever for as long as it takes.

The model for the various FD polling mechanisms is "wake me up when read(2) won't return EAGAIN when I call it later". That is very natural for sockets, because what read(2) is doing is draining a kernel side receive buffer, so the question is "wake me up when the buffer isn't empty". If the FD were a file on disk, knowing ahead of time that read(2) isn't going to return quickly is kind of a clumsy task.

Re:Efficient IO with io_uring [pdf] by bogomipz on Tuesday May 07, 2019 @11:38AM (#19850508)
>"If the FD were a file on disk, knowing ahead of time that read(2) isn't going to return quickly is kind of a clumsy task."

But isn't this exactly what happens? A process running in kernel mode calls schedule() and sleeps waiting for I/O. The reason for calling schedule() is that it already does know read isn't going to return quickly. But maybe I'm completely misunderstanding your explanation about why current AIO on Linux is clunky?

Re:Efficient IO with io_uring [pdf] by asveikau on Tuesday May 07, 2019 @11:48AM (#19850589)
The key observation is this ahead of time thing. If you've reached the point of invoking the scheduler you have already done a bunch of work to decide that, based on a request that has already started. On the other hand if epoll_wait returned and said the fd was ready, but the read hasn't yet been issued, who is to say the conditions will be the same later? Keep in mind also that other processes can do things that evicts caches, or make other requests to the same disk or controller.

For all of these cases I suppose you could make the I/O fail with EAGAIN, but this seems like a lot of potential for spurious wakeups.

Re:Efficient IO with io_uring [pdf] by bogomipz on Tuesday May 07, 2019 @03:49PM (#19852818)
Thanks for the clarification, that makes good sense. Cheers.
Re:Efficient IO with io_uring [pdf] by snarfy on Monday May 06, 2019 @07:55PM (#19845115)
Most of the problems with async i/o were spelled out very well in the recent "I/O is Faster Than CPU" [1] posted a few days ago.

[1] - https://news.ycombinator.com/item?id=19818899

The ring buffer is definitely innovation.

Re:Efficient IO with io_uring [pdf] by crest on Monday May 06, 2019 @07:20PM (#19844916)
Looks like a little bit of both. Linux has a lot to catch up because epoll is a mess. It is limited to file descriptors with a readiness semantic (sockets, pipes, ttys) meaning that it will never work with normal files because they are always ready while blocking whenever they feel like it. The second API fuck up is the fork/exec semantic. Oh and they tried to get away with edge triggered instead of level triggered. All in all epoll is a just a nasty case of NIH. The epoll developers just wrote code without understanding the prior art in their field (e.g. kqueue).
Re:Efficient IO with io_uring [pdf] by oralty on Monday May 06, 2019 @07:46PM (#19845067)
epoll (and even kqueue) is not aio and was never intended as such. While epoll had some early snafus, those have long been fixed (offering ET and LT) and it works well for its intended purpose (io readiness). This (io_uring) is more for a thing like Windows overlapped IO and "unix" (incl FreeBSD) aio. AIO on linux is yet a separate set of syscalls (io_*) and has since day one in the 2.5 days been pretty much a useless joke, very limited in scope (O_DIRECT only).

kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html.

Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.

Re:Efficient IO with io_uring [pdf] by gpderetta on Monday May 06, 2019 @08:10PM (#19845210)
The link doesn't explain much. The only argument that Linus has on kqueue is that is overengineered. Then it goes on a long multi message rant in how having multiple queues is insane.

Luckily, in the end epoll ended up supporting multiple queues at least.

I do believe that ET is good though.

Re:Efficient IO with io_uring [pdf] by AnanasAttack on Monday May 06, 2019 @07:39PM (#19845034)
What do you dislike about edge triggered mode?