A computer may need to read the data millions of times, but humans only need to read it when things goes wrong. If you take the route of using a binary format, then you need to provide a robust set of tools to view, edit, debug that format. It can absolutely work, journald is binary, but has the tooling to help you get the plain text, when you need it. Protobuf could work well, but if you design your own binary data format you have to build some tooling as well.
For many (most) it doesn't matter much if you use XML or JSON, the computer will process it fast enough. If the concern is parsing speed, then yes, absolutely binary formats might be the best option. For things like HFT or anything else that processes vast volumes of data, its probably the only choice.
My background is being woken up at 3AM by alert of call for systems where I have less that 100% familiarity and is asked to troubleshot. Looking at a hexdump for a format I don't know is not all that helpful. Seeming a mangled XML document, or reading the JSON output from an API call can frequently be enough to know if I need to call a developer, or if I can work with the client and fix the data.
Binary formats are absolutely a valid solution of a number of problems and if it works in your context there's no reason to change it.
I recently had the joy of reverse engineering a terrible piece of JavaScript which created a vast and ridiculously complicated JSON delivery for one of our most important frontends. It was so full of any assumptions that the original author had several functions which were still doing various things with “unknown” because the data which had once been there wasn’t. Anyway, since I had to reverse engineer it from terrible code and no documentation I figured I’d just talk with the people who worked the data inputs and outputs and maybe learn something about the process from their business processes. Turns out they couldn’t help, what was even worse was how the frontend partner we were delivering the JSON to couldn’t even explain why they needed it the way it arrived. Those were some “fun” sessions trying to Sherlock Holmes out of the mess. This was a bad story, but it’s just one out of a thousand such stories in enterprise and non-tech IT.
I think the reason JSON “won” over XML was the ”same” human reason. I’ve seen a lot of XML, but I don’t think I’ve ever seen some that didn’t break its own standards. Like you’d get all XML with data between ><‘s and then suddenly a single field would have the data inside the <… value=“”>… because duck you, or whatever else. Often they wouldn’t bother supplying you with schemas, or if they did, their auto-generated schemas wouldn’t have been updated in five years because apparently someone forgot that. Then once you “progress” beyond the technical funzies you’ll still have to deal with using the XML to find out what the hell is going on, and since it’s much harder to read (not entirely sure why) you’ll have to first convert it into JSON anyway, before whatever business process person has to read it.
More like, the people who didn't understand XML was designed to simplify HTML's SGML serialization moved elsewhere to misuse JSON (and YAML, and other things), the use cases and limitations of which they also failed to understand.
From the XML spec (edited by Tim Bray, the author of the linked blog):
> XML is a subset of SGML that is completely described in this document. Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML. [1]
Many modern serialisation / deserialisation libraries (like Rust's serde) let you serialise your data structures to a variety of formats, including both JSON and various binary formats of your choice.
But it should also be relatively easy to take any binary format, and define an alternate text based format for it that allows one-to-one conversion in both directions. A bit like assembly vs binary machine language.
Biggest issue with binary is the difficulty reading / manipulating it long-term.
In my opinion, one should go for XML when writing a portable document format of any variety to allow a vast array of schema validators, linters, and so-forth. This makes writing plugins which target a given schema much more portable and easier to write.
It's kind of ridiculous how many times web folks reinvent the wheel. Seriously, get rid of YAML, TOML, JSON. We already had INI, XML/XPATH/XSD.
There's nothing wrong with XML, people are just lazy and so is JSON. It's the lazy, sloppy cousin of XML which was a well-thought-out standard to fill a deficiency in HTML for data.
Honestly though, we should be using something like this: http://openddl.org/
I agree with you on YAML/JSON, but isn't TOML more or less INI with more data types and an actual specification?
And to be fair, YAML is somewhat clean for what it is. I shouldn't have included that in my rant either. My main gripe is with the "JSON Everywhere" approach. :)
There's no one-size-fits-all solution to anything. If one is using JavaScript—great, use JSON as a way to transmit objects over the wire or unpack an object into a template. Just PLEASE do not use it for a configuration format. Visual Studio Code configuration is a nightmare of ugliness.
INI is not a defined language. TOML is the most popular attempt to define it.
XML doesn't really encode the same information as YAML and JSON. (It encodes more, but what is important, it encodes it differently.)
Exchanging your things on the left with your things on the right is guaranteed to make a day turn bad.
Now 18 years later, JSON seems to have displaced it.
Personally, I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; I much prefer simple and efficient binary formats, which can be just as extensible without the additional inefficiency and needlessly-introduced-failures of string handling.