News for hackers. Stuff that matters.
Re:What's So Bad About Posix I/O? by exDM69 on Tuesday September 12, 2017 @05:03AM (#15225837)
You can't do that with normal read/write and select/poll/epoll, but you can either use aio (posix) or io_submit (linux-specific).

In my opinion, a big issue with these is that asynchronous i/o requests can not be used with select/poll/epoll, but rely on other notification methods instead. AIO can use unix signals (ugh) or thread notification (see sigevent(7)) to signal completion. io_submit uses io_getevents to read events about completed I/O requests.

From a userspace programmer's point of view, it would make much more sense if io_setup() would return a file descriptor that could be used with select/poll, and io_events would be read() from the fd.

Currently it's impossible (?) to have filesystem i/o in the same select/poll loop with your network i/o because of this issue. This is why libuv falls back to using threads and synchronous i/o to provide "async" i/o.

Re:What's So Bad About Posix I/O? by derefr on Tuesday September 12, 2017 @05:51AM (#15226004)
> Currently it's impossible (?) to have filesystem i/o in the same select/poll loop with your network i/o because of this issue.

Hacky workaround: don't do filesystem i/o in the first place. Have the kernel serve an iSCSI target, then consume it over the loopback interface entirely in userspace. Your disk now has network semantics! (But you have to bring your own filesystem driver.)

If you don't mind some extra abstraction overhead, you can skip past iSCSI and instead build a userspace NFS client lib into your app, consuming a loopback NFS share. s/NFS/9P/ if your disk server has in-kernel 9P.

(Is there any use to these approaches? Yes: they're actually nearly-optimal ways to persist state from a unikernel running on a hypervisor, because all the disks are remote one way or another anyway. Erlang on Xen uses the 9P approach, for example.)

Re:What's So Bad About Posix I/O? by sitkack on Tuesday September 12, 2017 @09:01AM (#15226898)
This aligns all semantics and removes file io as special case? Unikernels appear to be a good way to evolve past Unix.
Re:What's So Bad About Posix I/O? by MichaelMoser123 on Tuesday September 12, 2017 @05:26AM (#15225919)
you can do io_submit from one thread and then have some dedicated threads do io_getevent + event signalling (comes at a price though).

Still all this event goodness is not good enough for things like infiniband and the brave world of zero copy networking.

Re:What's So Bad About Posix I/O? by exDM69 on Tuesday September 12, 2017 @06:05AM (#15226062)
Yeah, sure that kinda works but it won't play nice with having an event loop on network events. A typical high performance server has multiple worker threads waiting on events on a select/epoll and if you add in another thread (or several) waiting on io_getevent you end up in a rather contrived solution.

You could use a pipe or eventfd or some other fd-based mechanism to wake up a worker thread on a filesystem event, but then you get two context switches (one to wake up io_getevent thread and another to wake up a worker). This carries a lot of indirect costs from jumping back and forth to the kernel.

I am unaware of a solution that would make filesystem i/o and network i/o play nice in one event loop without adding some overhead.