News for hackers. Stuff that matters.
Re:Varlink – IPC to replace D-Bus gradually in systemd by userbinator on Sunday September 29, 2024 @09:49PM (#41692683)
It doesn't matter what the rate is.

Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd.

[1] Maybe XML would be a close competitor.

Re:Varlink – IPC to replace D-Bus gradually in systemd by Spivak on Sunday September 29, 2024 @11:26PM (#41693251)
It's the lingua-franca serialization format that has bindings to every programming language already, doesn't require agreeing on the payload layout beforehand, is beyond overkill for the task being asked of it, and is human debuggable over the wire.

I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by comparison. yyyyuua(yv) It's like the printf message protocol but worse.

Re:Varlink – IPC to replace D-Bus gradually in systemd by quotemstr on Sunday September 29, 2024 @11:47PM (#41693355)
JSON isn't simple. It's simplistic. It's too simple. By refusing to acknowledge the complexity of the real world at the protocol layer, varlink shifts the burden of dealing with the complexity from one transport layer to N ad hoc application layers, making us all worse off.
Re:Varlink – IPC to replace D-Bus gradually in systemd by Spivak on Monday September 30, 2024 @12:00AM (#41693430)
Do you mean https://varlink.org/Interface-Definition? JSON is just the marshaling format. All the language bindings don't need a fancy parser outside of json.loads. I wish the interface definitions themselves had been specified in JSON as well.

Peep the Python interface parser: https://github.com/varlink/python/blob/master/varlink/scanne.... This is the quality you can expect for an ad-hoc format, which is to say, pretty bad. Making the hardest part farming out to a parsing library you already have and is a built-in in your language is a no-brainer choice.

Re:Varlink – IPC to replace D-Bus gradually in systemd by foul on Monday September 30, 2024 @08:29AM (#41696398)
Being simplistic is a leitmotiv of the project, at least someone in the post-GNOME cabal stopped reinventing the wheel.
Re:Varlink – IPC to replace D-Bus gradually in systemd by dathinab on Monday September 30, 2024 @03:40AM (#41694545)
> lingua-franca serialization format

it isn't, fundamentally can't be as it has no support for binary blobs

and base64 encoded strings are a pretty terrible solution for this

and the space overhead of many small number can also be pretty bad

and while a lot of IPC use cases are so little performance sensitive that JSON is more then good enough and large blobs are normally not send over messages (instead you e.g. send a fd) there are still some use-cases where you use IPC to listen to system events and you have a situation where a pretty insane amount of them is emitted, in which case JSON might come back and bite you (less due to marshaling performance and more due to the space overhead which depending on the kind of events can easily be 33%+).

But what I do not know is how relevant such edge cases are.

Probably the biggest issue might be IPC system mainly used on 64bit systems not being able to properly represent all 64 bit integers .... but probably that is fine too.

Re:Varlink – IPC to replace D-Bus gradually in systemd by orf on Monday September 30, 2024 @04:58AM (#41694986)
The protocol supports “upgrading” requests. If your service relies on sending large binary blobs (over this? Why?), then it doesn’t have to be done with JSON.

For example, the metadata of the blob could be returned via JSON, then the request is “upgraded” to a pure binary pipe and the results read as-is.

Re:Varlink – IPC to replace D-Bus gradually in systemd by dathinab on Monday September 30, 2024 @07:51AM (#41696115)
not one large blob, many small events

binary blobs is just the biggest example and was only mentioned in relation to the "lingua franca" argument, many other common things are also larger in JSON. Only if you have many larger not escaped utf-8 strings does this overheads amortize. E.g. uuids are something not uncommonly send around and it's 17 bytes in msgpack as a bin value and 38 bytes in json (not inlcuding `:` and `,` ). That 211% the storage cost. Multiply it with something going on and producing endless amounts of events (e.g. some unmount/mount loop) and that difference can matter.

Through yes for most use cases this will never matter.

Re:Varlink – IPC to replace D-Bus gradually in systemd by orf on Monday September 30, 2024 @08:24AM (#41696352)
I get your point, but you have to understand that for every second you’ve spent writing that comment, globally hundreds of millions of HTTP responses have been processed that contain UUIDs of some kind.

Yes, there’s a more optimal format than hex-encoding UUID values. However it simply does not matter for any use case this targets.

16 bytes vs 38 bytes is completely meaningless in the context of a local process sending a request to a local daemon. It’s meaningless when making a HTTP request as well, unfortunately.

I’d have loved Arrow to be the format chosen, but that’s not lowering the barrier to entry much.

Re:Varlink – IPC to replace D-Bus gradually in systemd by dathinab on Monday September 30, 2024 @08:42AM (#41696501)
it really isn't irrelevant in edge cases

sure most people don't write code for such edge cases, like ever

but e.g. systemd does sometimes because they reliable appear when running things at e.g. googles scale

Re:Varlink – IPC to replace D-Bus gradually in systemd by petre on Monday September 30, 2024 @11:51AM (#41698575)
> JSON is beautiful, elegant, and minimal by comparison.

True, but binary formats like CBOR or MessagePack are also that and efficient and have support for binary strings. This JSON is human readable thing is the nerverending argument of high level programmers. Anyone can read hex given enough time.