I like to joke that I invented ndjson over a decade ago when I accidentally forgot to put things in an array before `json.dumps`, I just wasn't smart enough to call it a standard. But when you do end up with ndjson when you wanted an array of results, or vice versa, JSON makes it easy to munge things to where you need.
Compare that to something like protobuf: it's not a self-synchronizing stream, so if you send someone multiple messages without framing them (prefix by length or delimited are popular approaches), they're going to decode a single message that doesn't make much sense on the other end. And they won't be able to fix it at all.
So I guess JSON is New Jersey style design[1].
FWIW, this is a conscious design decision with Protobuf: it allows for easy upsert operations on serialized messages by appending another message with the updated field values. This is very useful for middleware that wants to either just add its own context to a message it doesn't even parse [1], or for middleware that might handle protobuf messages serialized with unknown fields.
On the other hand, 'newline delimited protobuf' is much less useful day-to-day than ndjson, as gRPC provides message streaming, which solves the issue of wanting to stream small elements of a long response (which is the general usecase of ndjson from my experience). For on-disk storage of sequential protobufs (or any other data, really), you should be using something like riegeli [2], as it provides critical features like seek offsets, compression and corruption resiliency.
[1] - eg. passing a Request message from some web server frontend, through request routers, logging, ACL and ratelimit systems up to the actual service handling the request.
Honestly if json had as part of its spec line-delimited arrays, and accepting NaN, it'd be close to perfect. Oh and native ints, but that is JS's problem.
Well, and a single, canonical spec. And a hard limit (however high) on nesting depth. And some other things. Ok, maybe it's far from perfect.
In the current world this seems like a lifestyle choice that sets yourself up for constant self-punishment.
I might be a curmudgeon but I'll take JSON for data interop any day over anything that _requires_ tooling (protobuf, gRPC). And I'll take it over the XML ecosystem too.
The faults of JSON seem, in practice, to be less harmful than the faults of other formats.
b) protobuf libraries and protoc have given me way more grief overall than json (python, js, c++)
If your workflow already supports it, I can see it being useful, but it's got a pretty steep learning curve to be honest, certainly more than json, despite the ill-implemented libs out there. If I wanted a binary format, IMHO I'd go for msgpack first, and reach for protobuf if that didn't work for me.
Protobuf (and flatbuffers) supports parsing messages from JSON instead of a binary blob. Best of both worlds IMO.
I personally would rather still go with Protobuf if I'm going to put in the effort to add a schema and codegen. It gives me other nice-to-have features (faster [de]serialization, smaller messages, field numbers and schema evolution, nicer IDL [not JSON!], gRPC, ...) and does away with some problems intrinsic to JSON that no schema system will fix (terrible number type, lack of binary type, slow parsing). It also has some interop with JSON in the rare case you absolutely positively need to convert to/from it (which is IMO the only upside of using JSON Schema in case you need that interop).
I still pine for binary-only data formats. They're easier to program, and nobody makes the mistake of trying to edit them manually or compose them in a shell script. Parsing data shouldn't be hard, but it also shouldn't be so easy that people hang themselves by accident.
Of course, the reason why we largely have text data formats is because it's insanely simpler to troubleshoot systems that use them. Some things should just be easier to manipulate. But for general purpose work, I miss binary data formats.
Zip is probably my favorite general-purpose binary data format. It's old, well defined, works with any kind of data, and you can immediately seek to data in very large archives rather than having to parse the entire thing first. And then there's that whole compression thing. If you wanted to distribute a thousand tiny blobs of CSV, JSON, YAML, and XML, all in one container, you could do much worse than Zip.
It manages to be both a poor data interchange language compared to JSON, and also a bad human-friendly langage due to the above ambiguities.
Unfortunately it's still the best human-friendly configuration language in wide use, so I use strictyaml (https://hitchdev.com/strictyaml/) instead.
I fully agree YAML is just...not good as a transport/interchange serde.
Personally, I actually really like HCL as a human-friendly config language, but it's got challenges in writing it, and thus support in most languages, if even present, is read-only.
Will look into strictyaml!
All of the problems you list of YAML are due to humans making assumptions, rather than having a program do all of the serialization/deserialization. YAML has a ton of useful data structures and they do not cause problems when you use a real parser to populate or interpret them. It's also a superset of JSON, so it's clearly not poor in comparison to JSON.
It's also not a configuration language. It's a data serialization format.
If you make something human-readable, humans will edit it. The only reliable way to prevent that is to make the serialisation format binary, IMO.
YAML also supports a number of really powerful features like you point out (tags, anchors etc.) but I don't see them widely used due to security risks with untrusted input, and interlanguage compatibilty problems. So personally I don't think it's a super good data serialisation language either, despite its power, at least compared to something like Protobuf.
As far as 'standards' go, literally nothing uses Gzip/DEFLATE outside of Linux machines and HTTP clients/servers. Gzip isn't even an archive format.
Like what?