News for hackers. Stuff that matters.
Re:Varlink – IPC to replace D-Bus gradually in systemd by arianvanp on Sunday September 29, 2024 @02:45PM (#41689448)
CBOR is the obvious choice here.

I've suggested it before and the systemd folks didn't seem completely opposed to it. Also because cbor parser is already in the dependencies due to FIDO2 disk unlocking

Re:Varlink – IPC to replace D-Bus gradually in systemd by theamk on Sunday September 29, 2024 @03:26PM (#41689712)
What is the data rate (messages, bytes) that you expect for this "D-Bus replacement protocol"?

Which fraction of CPU will JSON ser/des will take to justify using CBOR?

Re:Varlink – IPC to replace D-Bus gradually in systemd by XorNot on Sunday September 29, 2024 @08:08PM (#41692072)
I mean conversely every second week on HN someone complains about "inefficient modern development practices" and this is the sort of minor thing which would be trotted out as an example.

So I'd argue it's not an unreasonable question (although I lean closer in: better types then JSON is the problem, since a decent text serialization format is always going to be needed for debugging and development).

Re:Varlink – IPC to replace D-Bus gradually in systemd by KerrAvon on Sunday September 29, 2024 @11:51PM (#41693379)
This sounds like it wants to be a fundamental IPC facility in Linux, which seems like a major thing.

Any binary format can be dumped to a text when viewed for debugging. That’s really a nonissue. Have the tools dump to JSON if you like.

Re:Varlink – IPC to replace D-Bus gradually in systemd by akira2501 on Sunday September 29, 2024 @06:09PM (#41691166)
I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it lead to bad library design which often leads to easily compromised code. Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in.

It gives you a lot of decent protocol wire design and then flatly ignores everything we've learned about these types of designs in the last 3 decades. Be on the lookout for client libraries to slowly add in all of these checks as the vulnerabilities are discovered in them.

Re:Varlink – IPC to replace D-Bus gradually in systemd by rkeene2 on Sunday September 29, 2024 @07:48PM (#41691974)
FWIW, ASN.1 BER supports an indefinite length encoding (though DER does not; ASN.1 JSON and ASN.1 XML do as well)
Re:Varlink – IPC to replace D-Bus gradually in systemd by akira2501 on Sunday September 29, 2024 @08:08PM (#41692075)
ASN suffers from some of the other problems regardless of the encoding.

I was thinking about this more and I think FTP actually had a lot of the right ideas here. You want two channels. One for fast interactive command messaging and a second one coordinate especially just for bulk transfers arranged over the command channel. The design flaw in FTP of putting these on two separate ports put it immediately in conflict with firewall best practices so I think it went mostly unnoticed that it fundamentally is a good arrangement.

What you want is one channel, sequenced packets, with a defined maximum message length that is negotiated at startup and never changes during the life of the channel. This should probably never be more than 65k. There should be a known length packet type, and an unknown length packet type, with any attempt to send more than the negotiated maximum triggering an error and disconnect.

If you do need to send more than the negotiated amount you should open a new connection, in a bulk transfer mode, that after the initial handshake, has no protocol and is merely an associated stream of bytes that are wholly uninterpreted by the middleware layer other than to help you associate it with the sequenced packet that requested its initiation.

You'd actually be able to use TCP (with DSCP even), mostly avoid head of line blocking and multiplexer latencies, and have a reasonable security guarantee in the sequenced packet mode, and never have a protocol which pretends that 4GB strings in the middle of a packet are necessary or even a good idea to "support."

The downfall of this is that it would be much harder to implement it on a serverless architecture and would be nearly as complicated as a protocol as WebSocket ends up being. It might be worth playing with as a concept anyways.

Re:Varlink – IPC to replace D-Bus gradually in systemd by rkeene2 on Sunday September 29, 2024 @09:51PM (#41692705)
I think the protocol on which PDUs are exchanged on is a bit orthogonal to the PDU protocol itself here, but the split-mode networking protocols (FTP, IRC DCC, mosh, SIP, etc) always do end up getting less used because establishing two different connections is a bigger burden and there really needs to be a great advantage to doing so, and often require other considerations (better connection tracking, TURN/ICE servers, etc).

For a localhost/UNIX domain protocol, it might work since most of these considerations are significantly reduced.

As to DSCP, it might be useful across controlled networks but in general, in my experience it's not really ever honored broadly.

Back on the original topic, when I was building this for my Linux distribution I ended up just using basic JSON with TLS (for client certificate authentication) though since having authentication for remote management was a goal, and once I was already having to perform a TLS negotiation then the PDU consideration for performance wasn't something to really spend too much time on.

Re:Varlink – IPC to replace D-Bus gradually in systemd by IgorPartola on Monday September 30, 2024 @12:43AM (#41693622)
You would need a pretty intense QoS system built for this if you are using something like TCP for this using two different connections. Underneath it would still use IPv4/6 packets and you only have so much bandwidth. The bulk transfer channel might stall the transfer of control packets temporarily.

You would need to tightly control and multiplex the messages yourself, needing to do this in one connection or using something like TCP priority flag.

Personally I just think that TCP is a shitty protocol for building applications. In almost every use case either UDP or SCTP are a better choice. With raw UDP datagrams you aren’t guaranteed delivery but it’s great for the kind of telemetry where the last message in is what matters. You can also build quite flexible stuff on top of it.

SCTP gives you in sequence reliable datagrams and congestion control. This means you don’t have to devise message length communication into your application layer protocol and can rely on your transport. It also has multiplexing built right in. If it had a built in checksum it would truly be ideal. Not sure if something like secure communication really belongs at this level but if it had that I double we would ever use anything else. From SCTP’s Wikipedia page:

> SCTP applications submit data for transmission in messages (groups of bytes) to the SCTP transport layer. SCTP places messages and control information into separate chunks (data chunks and control chunks), each identified by a chunk header. The protocol can fragment a message into multiple data chunks, but each data chunk contains data from only one user message. SCTP bundles the chunks into SCTP packets. The SCTP packet, which is submitted to the Internet Protocol, consists of a packet header, SCTP control chunks (when necessary), followed by SCTP data chunks (when available).

Re:Varlink – IPC to replace D-Bus gradually in systemd by Netch on Monday October 07, 2024 @07:38AM (#41764998)
> SCTP

> It also has multiplexing built right in.

It is not. You canʼt stop, for example, receiving from stream 0 during getting high-priority data from stream 9. No userlevel API allows to specify "now receive a message exactly from stream 9". Data packet TSNs are the same number sequence for all streams. You canʼt normally ignore a single data TSN for a stream payload: you have to issue SACK and brook constant retransmissions from other side. You canʼt specify receive window separately for each stream, to calm the sender down while the streamʼs receive buffer is full.

I donʼt know who and why spreads the myth it is really multiplexing, but now these stream numbers are merely another type of per-message external tag. All other is merely scam. It seems that initial design was moving toward this possibility but then something unfortunate happened.

> This means you don’t have to devise message length communication into your application layer protocol and can rely on your transport.

This is, well, tasty from it.

Re:Varlink – IPC to replace D-Bus gradually in systemd by xorcist on Monday September 30, 2024 @02:44AM (#41694223)
One channel, multiplexing, sequenced packets. It looks like you are describing SSH.
Re:Varlink – IPC to replace D-Bus gradually in systemd by gjvc on Sunday September 29, 2024 @08:27PM (#41692178)
"lead to bad library design"

how?

Re:Varlink – IPC to replace D-Bus gradually in systemd by akira2501 on Sunday September 29, 2024 @09:19PM (#41692501)
More complex buffer management in non-garbage collected languages.

man 3 cmsg

For a glimpse into the darker corners of hell. At least you get to grip it there. In GCed languages you have to hope the library designer gave you reasonable controls for timeouts, aborting large data transfers, and releasing buffers efficiently in a bursty network environment.

See all the controls any HTTP server gives you for this. "Maximum header size", "Maximum header(s) size", "Maximum Body Size", "Request Header Timeout", "Total Request Timeout."

None of those were added by default or by tasteful library authors. They were all added to answer to specific emergent security vulnerabilities that were discovered and then were demanded by the users.

Re:Varlink – IPC to replace D-Bus gradually in systemd by XlA5vEKsMISoIln on Monday September 30, 2024 @01:58AM (#41693995)
JSON's everything is "indefinite length". Also its implementations are wildly inconsistent [1] (is anyone going to read the code to figure out which is the blessed systemd parser?). Also it doesn't have integers. A lot of things with JSON will deteriorate into stringifying anyway, for example dates.

[1]: https://seriot.ch/projects/parsing_json.html

Re:Varlink – IPC to replace D-Bus gradually in systemd by trissylegs on Monday September 30, 2024 @03:23AM (#41694430)
CBOR does have indefinite length arrays, maps, byte strings and text strings : https://www.rfc-editor.org/rfc/rfc8949.html#name-indefinite-...
Re:Varlink – IPC to replace D-Bus gradually in systemd by Retr0id on Monday September 30, 2024 @11:46AM (#41698515)
I'm a fan of DAG-CBOR, which is a deterministic subset of CBOR with most of the weird bits excluded (like indefinite lengths, multiple floating point formats, etc.), but it can still represent any sensible JSON object.
Re:Varlink – IPC to replace D-Bus gradually in systemd by Netch on Monday October 07, 2024 @04:42AM (#41763961)
> I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it

CBOR is not. No full message length. Only primitive type values. Compound type values require length _in items_ (not bytes) but indefinite encoding is possible if allowed ("deterministic" constraint to message forbids it).

So technically this doesnʼt differ from e.g. JSON where you have to wait until final "}" or "]" for a long compound item, and have to install an artificial limit.

> Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. > As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in.

So does any such a protocol. JSON, XML, any ASN.1, CBOR, whatever. If there are no extensible tags, there are field names - or they will be added by customer using what is allowed. Iʼve seen this in ASN.1 sequence-of pairs of name+value, directly emulating JSON-like dictionary. To limit it is not how security issues are handled.

Re:Varlink – IPC to replace D-Bus gradually in systemd by formerly_proven on Sunday September 29, 2024 @06:19PM (#41691290)
That'd be "I can't believe it's not msgpack with extra design flaws and the name changed to my own name" CBOR?
Re:Varlink – IPC to replace D-Bus gradually in systemd by nly on Sunday September 29, 2024 @06:54PM (#41691570)
It's not the data format that is important, but the tooling, code generation and binding quality.
Re:Varlink – IPC to replace D-Bus gradually in systemd by KerrAvon on Sunday September 29, 2024 @11:48PM (#41693362)
Data format is absolutely important for operating system IPC. JSON is raw text. Messages occupy more RAM than a binary format, which increases your working set, and it requires at least some nontrivial parsing. This ultimately limits the number of messages you can send per second.
Re:Varlink – IPC to replace D-Bus gradually in systemd by jcelerier on Monday September 30, 2024 @06:14AM (#41695463)
It's not that simple:

    [1.2,1.3,1.4]
is 13 bytes, the equivalent binary storage is 3*sizeof(double) (24 bytes) + whatever overhead exists for representing a dynamic array in your language
Re:Varlink – IPC to replace D-Bus gradually in systemd by jchw on Monday September 30, 2024 @08:10AM (#41696265)
Personal opinion: don't put floating point numbers into an API. If you really want real numbers passing them as decimal string is actually sensible. This can be done without using JSON, of course. Can also do fixed precision as well, which will store reasonably well into a VLQ.

OTOH this is pretty cherry picked. It is questionable if you really need floating point numbers if your numbers are all this small and low precision, but in actuality unless this comprises your entire message, you'll probably still lose to JSON overhead eventually anyways, whereas non-self-describing serialization like capnp has effectively no overhead by default. The advantage of encoding as doubles is that it is predictable and won't explode into taking massively more, its always the same size per number. If you want it to be smaller for some reason, compression is always an option, though I suspect for IPC it's the wrong thing to be concerned about.

Re:Varlink – IPC to replace D-Bus gradually in systemd by Netch on Monday October 07, 2024 @06:25AM (#41764562)
> If you really want real numbers passing them as decimal string is actually sensible.

Hexadecimal, if so. Nearly any decent modern language and runtime can it.

    $ python
    >>> import math
    >>> math.pi.hex()
    '0x1.921fb54442d18p+1'
This gives exact value without any rounding effects and conversion cost.

(I donʼt count decimal floating-point as it is quite rare and its application is faint. In overwhelming most of cases, fixed point is better.)

Re:Varlink – IPC to replace D-Bus gradually in systemd by gpderetta on Monday September 30, 2024 @07:33AM (#41695979)
If you expect to routinely send small numbers your serialization protocol can use floats or some small fix point format.

Variable length binary number encodings are also an option.

The dynamic array overhead in a language is immaterial when discussion serialization formats.

Re:Varlink – IPC to replace D-Bus gradually in systemd by immibis on Monday September 30, 2024 @07:10AM (#41695802)
What's the chance the data you want to transmit is exactly 1.2 rather than 1.205338843375?
Re:Varlink – IPC to replace D-Bus gradually in systemd by rkeene2 on Sunday September 29, 2024 @07:48PM (#41691969)
Or an ASN.1 DER encoded sequence -- ASN.1 is more common than CBOR (being in a lot of PKI stuff) and there are already generators for DER (or JSON or XML...) for a given schema.