News for hackers. Stuff that matters.
Re:Flow-IPC: Open-Source Toolkit for Low-Latency Inter-Proce... by signa11 on Friday April 12, 2024 @03:24AM (#40010197)
i have done something exactly identical at my current place of employment, and am always inquisitive to see how others have 'stacked-da-cat'.

we _unfortunately_ gravitated towards protobuf's despite my fervent appeal to go with capn-proto. that has caused a cascade of troubles / missed opportunities for optimizations etc. etc.

Re:Flow-IPC: Open-Source Toolkit for Low-Latency Inter-Proce... by fwsgonzo on Saturday April 13, 2024 @01:39PM (#40024655)
I tried to migrate to capn-proto but it just doesn't build on MinGW so I have no choice but to wait. Like you say, it gets worse the more I wait. But, if the APIs are somewhat sane, they should hopefully also be somewhat similar: Able to switch case on oneofs, movable data structures etc.

I don't like that protobuf has recently started linking with abseil, which despite being a good framework, I can't use it if it doesn't build absolutely everywhere I need it to. So, maybe I'll be forced over to CapnProto one of these days?

Re:Flow-IPC: Open-Source Toolkit for Low-Latency Inter-Proce... by Arelius on Saturday April 13, 2024 @07:35PM (#40027198)
Honestly I like capn-proto. But it's community is a bit light. Even Windows which is "supported" has been a bit fix-it-yourself.

If MinGW isn't already supported, if probably expect to have to take that over yourself.

Re:Flow-IPC: Open-Source Toolkit for Low-Latency Inter-Proce... by sgtnoodle on Saturday April 13, 2024 @06:25PM (#40026720)
I've also developed a strikingly similar low latency real-time IPC message bus for work. It also uses sockets with transparent shared memory optimization. In my case, it's the backbone for an autonomous aircraft's avionics. I made everything agnostic to the message scheme, though, and most of the tooling supports an in-house schema, protobuf, JSON, YAML, etc. There's also clients implemented in C++, Rust, Python and Julia.

What troubles has protobuf caused you?

Re:Flow-IPC: Open-Source Toolkit for Low-Latency Inter-Proce... by signa11 on Monday April 15, 2024 @07:07AM (#40038937)
> What troubles has protobuf caused you?

we are primarily a c++ shop, so all comments are from that vantage point. in no particular order following:

     1. extremely bloated generated code.

     2. wire format is not optimal, specifically for large nested messages which trashes the cache on a already severely constrained h/w.

     3. not easy to perform shared-memory exchange for large messages, for example, if you have 'string' types in your messages, those cannot be allocated on arenas.

     4. <this space is for rent>