D-Bus is just a bus so it can solve a bunch of different IPC problems at the same time. e.g. D-Bus can handle ACLs/permissions on RPC methods, D-Bus can handle multiple producers and multiple consumers, and so on. I think ultimately all that's really needed is IPC, and having a unified bus is just to allow for some additional use cases that are harder if you're only using UNIX domain sockets directly.
If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel.
Not that I think Varlink is any better. It seems like at best it's probably a lateral move. I hope this does not go forward.
This is my point.
My favourite DBus situation a number of years ago was a CentOS 7 box that reboot stopped working on with a cryptic DBus error that no one has ever seen before. I had to sync it, power cycle the node from the ILO card and cross my fingers.
I really don't give a shit about this. I just wanted to run my jobs on the node, not turn into a sysadmin due to someone else's dubious architectural decisions.
It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead…
> Public varlink interfaces are registered system-wide by their well-known address, by default /run/org.varlink.resolver. The resolver translates a given varlink interface to the service address which provides this interface.
…we have well known names, and sandboxing, or replacing a service for just one client, remains a mess. Sigh.
> It would be quite straightforward to map a capability-like UNIX socket into each service’s filesystem and give it a private view of the world. But instead…
Can you link to your PR where you solved the problem?
And OK fine. It's not that bad, most C ABIs are able to work around this reasonably OK (not all of them but sure, let's just call it a skill issue.) But then what do you do when you want anything more complicated than a completely fixed-size type? Like for example... a string. Or an array. Now we can't just use a struct, a single request will need to be split into multiple different structures at the bare minimum.
And plus, there's no real reason to limit this all to the same machine. Tunneling UNIX domain sockets over the network is perfectly reasonable behavior and most* SSH implementations these days support this. So I think scoping the interoperability to "same machine" is unnecessarily limiting, especially when it's not actually hard to write consistent de/serialization in any language.
* At least the ones I can think of, like OpenSSH[1], Go's x/crypto/ssh[2], and libssh2[3].
[1]: https://www.openssh.com/txt/release-6.7
[2]: https://pkg.go.dev/golang.org/x/crypto/ssh#Client.ListenUnix
You are adding more problems that don't exist to the specification.
As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
D-Bus does in fact already have support for remoting, and like I said, you can tunnel it today. I'm only suggesting it because I have in fact tunneled D-Bus over the network to call into systemd specifically, already!
> As for strings, just shove a char[4096] in there. Use a bit of memory to save a lot of parsing.
OK. So... waste an entire page of memory for each string. And then we avoid all of that parsing, but the resulting code is horribly error-prone. And then it still doesn't work if you actually want really large strings, and it also doesn't do much to answer arrays of other things like structures.
Can you maybe see why this is compelling to virtually nobody?
For the love of God, use a proper slice/fat pointer, please.
Switching over to slices eliminates 90%+ of the issues with using C. Carrying around the base and the length eliminates a huge number of the overrun issues (especially if you don't store them consecutively).
Splitting the base and the offset gives a huge amount of semantic information and makes serialization vastly easier.
Correct as you may be though the argument here is that you should just write raw structs into Unix sockets. In this case you can't really use pointers. So, realistically, no slices either. In this context a fixed-size buffer is quite literally the only sensible thing you can do, but also, I think it's a great demonstration of why you absolutely shouldn't do this.
That said, if we're willing to get rid of the constraint of using only one plain C struct, you could use offsets instead of pointers. Allocate some contiguous chunk of memory for your entire request, place struct/strings/etc. in it, and use relative offsets. Then on the receiver side you just need some fairly basic validation checks to ensure none of the offsets go out of bounds. But at that point, you've basically invented part of Cap'n'proto, which begs the question... Why not just use something like that instead. It's pretty much the entire reason they were invented.
Oh well. Unfortunately the unforced errors of D-Bus seem like they will lead to an overcorrection in the other direction, turning the core of our operating system into something that I suspect nobody will love in the long term.
The only problem I have with Cap'n Proto is that the description is external to the serialization. Ideally I'd like the binary format to have a small description of what it is at the message head so that people can process messages from future versions.
ie. Something like: "Hmm, I recognize your MSG1 hash/UUID/descriptor so I can do a fast path and just map you directly into memory and grab field FD. Erm, I don't recognize MSG2, so I need to read the description and figure out if it even has field FD and then where FD is in the message."
In general there's very little you can really do with a dynamic schema. Perhaps you can convert the message to JSON or something. But if your code doesn't understand the schema it received, then it can't possibly understand what the fields mean...
It's better to design APIs to be extensible in ways that doesn't require dynamic introspection. It's always possible to have a "generic" header message that contains a more specific message inside of it, so that some consumers of an API can operate on messages even when they contain some data that they don't understand, but I think this still warrants some care to make sure it's definitely the right API design. Maybe in the future you'll come to the conclusion it would actually be better if consumers don't even try to process things they're not aware of as the semantics they implement may some day be wrong for a new type of message.
Versioning, at the least, is extremely difficult without this.
Look at the Vulkan API for an example of what they have to do in C to manage this. They have both an sType tag and a pNext extension pointer in order for past APIs to be able to consume future versions.
In Lustre RPC _control_ messages go over one channel and they're all a C structure(s) with sender encoding hints so the receiver can make it right, and any variable-length payloads go in separate chunks trailing the C structures.
Whereas bulk _data_ is done with RDMA, and there's no C structures in sight for that.
Capnp sounds about right for encoding rules. The way I'd do it:
- target 64-bit architectures
- assume C-style struct packing
rules in the host language
(but not #pragma packed)
- use an arena allocator
- transmit {archflags, base pointer, data}
- receiver makes right:
- swab if necessary
- fix interior pointers
- fail if there are pointers
to anything outside the
received data
- convert to 32-bit if the
receiver is 32-bit
(That's roughly what Lustre RPC does.)As for syntax, I'd build an "ASN.2" that has a syntax that's parseable with LALR(1), dammit, and which is more like what today's devs are used to, but which is otherwise 100% equivalent to ASN.1.
Having actual interior pointers means not having to deal with pointers as offsets when using these objects. Now the programming language could hide those details, but that means knowing or keeping track of the root object whenever traversing those interior pointers, which could be annoying, or else encoding an offset to the root and an offset to the pointed-to-item, which would be ok, and then the programming language can totally hide the fact that interior pointers are offset pairs.
I've a feeling that fixing up pointers is the more interoperable approach, but it's true that it does more memory writes. In any case all interior pointers have to be validated on receiving -- I don't see how to avoid that (bummer).
What a fun sub-thread.
Because traditional POSIX IPC mechanisms are absolute unworkable dogshit.
> It is one of the worst serialisation decisions we ever made as a society.
There isn't really any alternative. It's either JSON or "JSON but in binary". (Like CBOR.) Anything else is not interoperable.
- Whatever: people in more niche languages are pretty used to needing to do FFI for things like this anyhow.
- Many of them already have a better ecosystem than D-Bus. e.g. interoperability between Protobuf and Cap'n'proto implementations is good. Protobuf in most (all?) runtimes supports dynamically reading a schema and parsing binary wire format with it, as well as code generation. You can also maintain backwards compatibility in these formats by following relatively simple rules that can be statically-enforced.
- JSON and JSON-but-binary have some annoying downsides. I really don't think field names of composite types belong as part of the ABI. JSON-like formats also often have to try to deal with the fact that JSON doesn't strictly define all semantics. Some of them differ from JSON is subtle ways, so supporting both JSON and sorta-JSON can lead to nasty side-effects.
Maybe most importantly, since we're not writing software that's speaking to web browsers, JSON isn't even particularly convenient to begin with. A lot of the software will be in C and Rust most likely. It helps a bit for scripting languages like Python, but I'm not convinced it's worth the downsides.
I pray to God nothing meaningful is actually doing what you are insinuating in any serious environment.
As for JSON, are you really that short sighted that it's the only method of encoding something? Is "oh well it doesn't fit the primitive types, so just shove it in a string and add another layer of parsing" acceptable? Hell no.
If you want something on the system level parsable by anything? Yes it is.
Edit: hell even XML is better than this!
Asn.1 is both quite complicated and not very efficient.
They could certainly have gone with protobufs or another binary serialisation format but would it really be better than the current choice?
I don’t think the issue they are trying to solve is related to serialisation anyway. Seems to me they are unhappy about the bus part not the message format part.
I can’t resist pointing that it’s basically a longer way of saying quite complicated and not very efficient.
That's very wrong. ASN.1 is complicated because it's quite complete by comparison to other syntaxes, but it's absolutely not inefficient unless you mean BER/DER/CER, but those are just _some_ of the encoding rules available for use with ASN.1.
To give just one example of "complicated", ASN.1 lets you specify default values for optional members (fields) of SEQUENCEs and SETs (structures), whereas Protocol Buffers and XDR (to give some examples) only let you specify optional fields but not default values.
Another example of "complicated" is that ASN.1 has extensibility rules because the whole "oh TLV encodings are inherently extensible" thing turned out to be a Bad Idea (tm) when people decided that TLV encodings were unnecessarily inefficient (true!) so they designed efficient, non-TLV encodings. Well guess what: Protocol Buffers suffers from extensibility issues that ASN.1 does not, and that is a serious problem.
Basically, with a subset of ASN.1 you can do everything that you can do with MSFT RPC's IDL, with XDR, with Protocol Buffers, etc. But if you stick to a simple subset of ASN.1, or to any of those other IDLs, then you end up having to write _normative_ natural language text (typically English) in specifications to cover all the things not stated in the IDL part of the spec. The problem with that is that it's easy to miss things or get them wrong, or to be ambiguous. ASN.1 in its full flower of "complexity" (all of X.680 plus all of X.681, X.682, and X.683) lets you express much more of your protocols in a _formal_ language.
I maintain an ASN.1 compiler. I've implemented parts of X.681, X.682, and X.683 so that I could have the compiler generate code for the sorts of typed holes you see in PKI -all the extensions, all the SANs, and so on- so that the programmer can do much less of the work of having to invoke a codec for each of those extensions.
A lot of the complexity in ASN.1 is optional, but it's very much worth at least knowing about it. Certainly it's worth not repeating mistakes of the past. Protocol Buffers is infuriating. Not only is PB a TLV encoding (why? probably because "extensibility is easy with TLV!!1!, but that's not quite true), but the IDL requires manual assignment of tag values, which makes uses of the PB IDL very ugly. ASN.1 originally also had the manual assignment of tags problem, but eventually ASN.1 was extended to not require that anymore.
Cavalier attitudes like "ASN.1 is too complicated" lead to bad results.
So, it's quite complicated. Yes, what I have been saying from the start. If you start the conversation by "you can define a small subset of this terrible piece of technology which is bearable", it's going to be hard convincing people it's a good idea.
> Cavalier attitudes like "ASN.1 is too complicated" lead to bad results.
I merely say quite complicated not too complicated.
Still, ASN.1 is a telco protocol through and through. It shows everywhere: syntax, tooling. Honestly, I don't see any point in using it unless it's required by law or by contract (I had to, I will never again).
> but it's absolutely not inefficient unless you mean BER/DER/CER, but those are just _some_ of the encoding rules available for use with ASN.1.
Sorry, I'm glade to learn you can make ASN.1 efficient if you are a specialist and now what you are doing with the myriad available encodings. It's only inefficient in the way everyone use it.
Subsets of ASN.1 that match the functionality of Protocol Buffers are not "quite complicated" -- they are no more complicated than PB.
> Still, ASN.1 is a telco protocol through and through.
Not really. The ITU-T developed it, so it gets used a lot in telco protocols, but the IETF also makes a lot of use of it. It's just a syntax and set of encoding rules.
And so what if it were "a telco protocol through and through" anyways? Where's the problem?
> It shows everywhere: syntax, tooling.
The syntax is very much a 1980s syntax. It is ugly syntax, and it is hard to write a parser for using LALR(1) because there are cases where the same definition means different things depending on what kinds of things are used in the definition. But this can be fixed by using an alternate syntax, or by not using LALR(1), or by hacking it.
The tooling? There's open source tooling that generates code like any XDR tooling and like PB tooling and like MSFT RPC tooling.
> Sorry, I'm glade to learn you can make ASN.1 efficient if you are a specialist and now what you are doing with the myriad available encodings. It's only inefficient in the way everyone use it.
No, you don't have to be a specialist. The complaint about inefficiency is about the choice of encoding rules made by whatever protocol spec you're targeting. E.g., PKI uses DER, so a TLV encoding, thus it's inefficient. Ditto Kerberos. These choices are hard to change ex-post, so they don't change.
"[T]he way everyone use it" is the way the application protocol specs say you have to. But that's not ASN.1 -- that's the application protocol.
There is no open source tooling that combines really used scheme understanding - in 5G this includes parameterized specifications by X.683 - and decoder able to show partially decoded message before an error, with per-bit explanation of rules led to its encoding.
> E.g., PKI uses DER, so a TLV encoding, thus it's inefficient.
When it is used, ~5% space economy is never worth people efforts to diagnose any problem. I strictly vote for this "inefficiency".
ASN.1 is not that complicated. Pity that fools who thought ASN.1 was complicated re-invented the wheel quite poorly (Protocol Buffers I'm looking at you).
Uh, no, structs, records, whatever you want to call them, are in many, if not most programming languages. "Structs" is not just "C structs" -- it's just shorthand for "structured data types" (same as in C!).
> Asn.1 is both quite complicated and not very efficient.
Every rich encoding system is complicated. As for efficiency, ASN.1 has many encoding rules, some of which are quite bad (BER/DER/CER, which are the first family of ERs, and so many thing ASN.1 == BER/DER/CER, but that's not the case), and some of which are very efficient (PER, OER). Heck, you can use XML and JSON as ASN.1 encoding rules (XER, JER).
> They could certainly have gone with protobufs or another binary serialisation format but would it really be better than the current choice?
Protocol buffers is a tag-length-value (TLV) encoding, same as BER/DER/CER. Having to have a tag and length for every value encoded is very inefficient, both in terms of encoding size as well as in terms of computation.
The better ASN.1 ERs -PER and OER- are much more akin to XDR and Flat buffers than to protobufs.
> I don’t think the issue they are trying to solve is related to serialisation anyway. Seems to me they are unhappy about the bus part not the message format part.
This definitely seems to be be the case.
It's plenty clear from discussion context that OP is talking about C struct but yes, replace C with any languages which suit you. It will still be part of the language semantic and not an IPC specification.
The point is you can't generally use memory layout as an IPC protocol because you generally have no guarantee that it will be the same for all architectures.
Certainly but that’s hardly structs anymore. You are implicitly defining a binary format which is aligned on the sender memory layout then.
I have had a misfortune to work for 5G which is full of PER-encoded protocols. Dealing with discrepancies in them - incompatible changes in 3GPP standard versions, different vendorsʼ errors, combined with usually low level of developers and managers in a typical corporation - was an utter nightmare.
IETF, in general, provides a good policy combining truly fixed binary protocols when they are unavoidable (IP/TCP/UDP levels) and flexible, often text, protocols where there is no substantial overhead from their use. Their early moves, well, suffered from over-grammaticalization (as RFC822). CBOR is nice here because it combines tagness and compactness. 3-bit basic tag combined with value (if fit) or length, it is commensurable with OER in efficiency but is decodable without scheme - and it is extremely useful in practice.
One requirement is being able to strace a misbehaving service and figure out quickly what it's sending and receiving.
This is a system-level protocol, not just a random user app.
As for JSON, please don't even start me on that. It is one of the worst serialisation decisions we ever made as a society. Poorly defined and unreliable primitive types, terrible schema support and expensive to parse. In a kernel, it does not belong! Give it a few months and varlink will be YAML over the top of that.