News for hackers. Stuff that matters.
Re:IPC in Rust – A Ping Pong Comparison by Sytten on Wednesday June 19, 2024 @12:15AM (#40724746)
It looks nice but the lack of async is sadly a non starter for a lot of us.
Re:IPC in Rust – A Ping Pong Comparison by mutatio on Wednesday June 19, 2024 @01:29AM (#40725134)
Is shared memory access naturally async like IO/io_uring? If not, asking for async is misplaced and any implementation would be synonymous with wrapping sync calls in `async fn` wrappers.
Re:IPC in Rust – A Ping Pong Comparison by dwattttt on Wednesday June 19, 2024 @02:41AM (#40725506)
I think it's as naturally async as anything else here actually. If you're calling a function via IPC over shared memory, you're sending a message, signalling it's there to the other process, and waiting for a response message with the result.

The only thing needed to turn this async is to change the "get result over IPC" function from "block until result is available" to "check if result is available, return TRY_AGAIN if it's not"

Re:IPC in Rust – A Ping Pong Comparison by tialaramex on Wednesday June 19, 2024 @05:26AM (#40726484)
In practice it's not desirable to go without an explicit wake feature, we need some way to know that now it's likely to succeed. The mechanism needn't be 100% reliable but "Always try again" is essentially exactly as useless as blocking.
Re:IPC in Rust – A Ping Pong Comparison by duped on Wednesday June 19, 2024 @10:27AM (#40728647)
No it's not, that's the whole point of using polling.
Re:IPC in Rust – A Ping Pong Comparison by loeg on Wednesday June 19, 2024 @11:16AM (#40729080)
I work on a widely deployed (~11,000 servers and growing) piece of software that does not use any OS sleep/wake primitive for our MPSC queues -- just polling. It works really well with the right overall design.
Re:IPC in Rust – A Ping Pong Comparison by bfrog on Wednesday June 19, 2024 @10:25AM (#40728629)
The signaling in this case is typically a shared atomic though not some OS primitive that a process can wake/epoll/etc on?
Re:IPC in Rust – A Ping Pong Comparison by elfenpiff on Wednesday June 19, 2024 @12:04PM (#40729550)
In iceoryx2, we use something that we call event concept, which can either use unix-domain sockets so that you have something to select/wake/epoll on or a variant where we use a semaphore stored in shared memory. See: https://github.com/eclipse-iceoryx/iceoryx2/tree/main/iceory...

The unix-domain socket has the advantage that you can combine it with external non-iceoryx events, but you pay with a small performance hit. The semaphore event is usually faster. As a user, you can configure your IPC service architecture and use the mechanism that suits you best (but I have to admit that we have not yet documented this in detail). For the zero_copy service variants, it is done here: https://github.com/eclipse-iceoryx/iceoryx2/blob/main/iceory....

Re:IPC in Rust – A Ping Pong Comparison by loeg on Wednesday June 19, 2024 @11:13AM (#40729053)
I believe you could do something with e.g. Linux futexes to support this kind of thing. But in general, polling at the receiver is going to be lower latency / higher throughput.
Re:IPC in Rust – A Ping Pong Comparison by loeg on Wednesday June 19, 2024 @11:12AM (#40729041)
The core primitive for IPC isn't "request and wait for a response," it's "send a single message and return immediately." Sure, if you want an RPC framework over IPC, maybe you want async blocking, but that's sort of a higher level concern than basic IPC.

It seems that iceoryx2 intends to support request-response in the future, but doesn't today:

> Request-Response (planned)

https://docs.rs/iceoryx2/latest/iceoryx2/

Re:IPC in Rust – A Ping Pong Comparison by elfenpiff on Wednesday June 19, 2024 @11:37AM (#40729265)
Here is one of the iceoryx2 maintainers, and we intend to implement request-response in Q3 this year.
Re:IPC in Rust – A Ping Pong Comparison by gpderetta on Thursday June 20, 2024 @03:20AM (#40735856)
Even with fire and forget, your send queue might become full. So you either drop the message of you have to wait for it to empty. Similarly on the receiving end you have to block waiting for the queue to become non empty.

In fact that's exactly the reason that async exists for networking.

Re:IPC in Rust – A Ping Pong Comparison by bfrog on Wednesday June 19, 2024 @10:24AM (#40728619)
What... its shared memory? What is asynchronous about touching a few atomic vars and shared memory (shared pages) exactly? You could absolutely poll on this sort of thing in a non blocking manner, but there is not OS primitive here to trigger a Waker
Re:IPC in Rust – A Ping Pong Comparison by gpderetta on Wednesday June 19, 2024 @12:30PM (#40729813)
build an Event Count on top of eventfd and you can get fast pathed blocking behavior on your shared memory queue that it is also async awaitable on top of your event loop of choice.
Re:IPC in Rust – A Ping Pong Comparison by bfrog on Wednesday June 19, 2024 @03:32PM (#40731647)
So add some syscalls and slowness to what should amount to a spsc atomic queue in memory? No thanks!
Re:IPC in Rust – A Ping Pong Comparison by gpderetta on Wednesday June 19, 2024 @05:27PM (#40732566)
The fast path can still be fast. You can make the blocking behaviour optional for those that need it.
Re:IPC in Rust – A Ping Pong Comparison by elfenpiff on Wednesday June 19, 2024 @11:39AM (#40729293)
If you want to use push notifications to wake up processes, iceoryx2 has the event messaging pattern where a listener waits for incoming events, and a notifier in another process can send specific events to all waiting listeners. See this example: https://github.com/eclipse-iceoryx/iceoryx2/tree/main/exampl...
Re:IPC in Rust – A Ping Pong Comparison by apatheticonion on Thursday June 20, 2024 @06:11PM (#40743986)
With this sort of thing you'd typically wrap the underlying IPC channel in a system thread and redirect messages to/from it within the local process via an async channel.

The dedicated system thread for the channel means the local process will never block on the incoming IPC events and the redirected messages will be buffered/enqueded as determined by the async runtime.

For reference, I wrote a utility that wraps an IPC library and exposes an async and sync API: https://github.com/alshdavid/ipc-channel-adapter