I am not sure if parsing XML is better than parsing JSON. Many languages or libraries' XML parser are dangerous by default. You usually need to manually configure your XML parser to be secure from XML-related attacks. Fortunately, some languages and libraries are going to make XML have a securer defaults, this is a good change. IMO, I think XML shouldn't have include many questionable features from security perspective.

For those unfamiliar with these attack vectors, there code injection and denial-of-service issues that in previous version of Python, were exploitable by default. Projects like https://pypi.org/project/defusedxml/ were designed to be secure against these issues by default, rather than requiring the library user to opt in.

The defusedxml project has an excellent matrix showing viability of the attack types against various python XML implementations: https://pypi.org/project/defusedxml/#python-xml-libraries

Agreed, and I like the libraries I saw in the past that deliberately only support a small subset of all XML extensions (sadly now I can't remember the names). Reducing attack surface and increasing sanity in one stroke is a policy that much more open-source software has to adopt.

> Many languages or libraries' XML parser are dangerous by default.

Seriously, XML External Entities is an incredibly dumb feature. To have it enabled by default makes it even worse.

You are right, and XML parsers can have a very large attack surface due to the sheer amount of specs to adhere to.

I see XML as better in the expressiveness it has, and more mature out of the box options to validate and transform it. Security and bugs remain an issue, but at the scale it can be used, there is a fighting chance to have experts dealing with the hardening of it all.

Swagger like format definitions are still pretty lax in my option in comparison. Now I wouldn't want to get back to XML land, I just think it occupies a pretty solid niche that is hard to match with anything more simple.

To be fair, XML wasn't intended as a data exchange format but as simplified SGML subset for use as delivery format on the web. While that largely hasn't happened, XML with XSD (sans rarely used feats) remains a strong exchange format for coarsely-grained inter-party traffic such as payment systems, taxes and other public/private data, etc.

I'm guessing the security deficits you mention are XML entity attacks. Well, SGML has CAPACITY ENTLVL in the SGML declaration to limit expansion depth. And a markup authoring or delivery format without entities/text macros is quite useless, even though HTML, when seen as a stand-alone markup language rather than SGML vocabulary, lacks it.

> XML wasn't intended as a data exchange format but as simplified SGML subset for use as delivery format on the web

You cannot deliver web content without... exchanging data.

And you cannot trust servers not to attack browsers.

> And you cannot trust servers not to attack browsers.

Interesting. I normally see it expressed the other way (trusting the server and not the client). Obviously, both are important.

There was a time when every browser hack made headlines, now it has to involve something big like specter to get even noticed. Browsers and browser based applications still feature heavily at Pwn2Own style competitions, but YOLO lets add more pointless features with bug ridden APIs with every browser update.

This is true and I remember when they were big headlines. I guess since I'm not in browser development, I'm used to the idea of contacting a known good server from my code, but my known good server being accessible from anyone on the internet.

Both are important, it's just interesting to see it written only from the other point of view.

Not wanting to disturb your browser sec discussion, but tbh I don't know what that has to with my GP post. If anything, SGML/XML allows hard validation of document data like basically no other mainstream tech.

I guess trust needs to be a two-way street. Even between computers.

Or lack-of-trust. Sanitize your inputs, because the other side is always an evil actor.

I would argue that, in the long run, gRPC/protobuf has a lower operational cost than JSON as long as you don't need to talk to it from a browser.

(Consuming it from a browser is a hassle because client-side JavaScript code is unable to speak the full gRPC API, so you need to fuss with reverse proxies to get everything working.)

What it doesn't have is a short learning curve. In order to get started, you need to learn the *.proto format, and how to use the code generator, and all the design implementations of the different data types it supports, and all of that.

But, once you get over that hump, it makes a lot of the hard stuff much, much easier to get right.

What I keep wishing for is some sort of "gRPC-lite" that doesn't include quite as many questionable micro-optimizations as protobuf/gRPC, but does include all of the really good ideas like specification-first API development, service reflection, and a clean logical separation between HTTP semantics and the semantics of the API that's being implemented on top of it.

> gRPC/protobuf has a lower operational cost than JSON

I'm a bit torn on this point.

We had a project where we needed to talk to an external grpc service, and protobuf was a technical given. Outside of the bootstrapping and typesetting, an issue we hit was the lack of tooling: Google officially supports protobuf libraries for 4 or 5 languages. There are other library providers, in particular for JS/Typescript, but from the get go there needs to be a strong library support or it's out of question. For instance looking at ruby, protobuf libs seem to be young at cursory glance.

It's true to some extent for JSON as well (though writing a parser for a limited set wouldn't be impossible), but I'd be hard press to come with a language that doesn't have a good JSON parser.

Then debugging also means relying on your library to encode/decode the payloads. If you need to replay a slightly modified query to check how an entry point reacts, you'll need to debug from your application, or build another test app just to emulate the requests. This issue gets compounded if you have doubt in your libraries' reliability. Development, maintenance, logging, monitoring etc. get that much heavier to deal with.

All in all I don't see protobuf as aiming for low operational cost. Or at least you need to be ready to bear the weight, and might be lucky if it ends up lighter than expected.

JSON can be scaled upward in reliability, with heavy testing and validation layers, while protobuf is hard to lighten.

That is a fair consideration. And part of what I was thinking about when I suggested that gRPC needs a slimmed-down alternative.

I think that part of the reason for weaker support on languages that Google doesn't personally maintain is that the gRPC protocol and tooling itself seems to be so technically complex that it's not really amenable to community support.

And I really don't think all that complexity is necessary. For example, in my own testing I found that the streaming protocol is generally not worth the extra hassle. Just getting rid of that would probably reduce the implementation cost considerably, and also eliminate the reason why client-side code can't speak gRPC.

I tried to get into gRPC/protobuf in Go on a Mac for a little hobby project, but man the effort just to get protoc up and running and then generate the stubs was insane. I'm sure somehow or another, it's user error, but when the barrier of entry is so high, it's hard to justify the effort when JSON-slinging is so rarely the bottleneck.

protobuf incurs lots of codebloat (especially with google's runtime / compilers) and the the serialization format is not really that nice IMO. I don't think it's possible to come up with 100% ideal format for all the use cases.

If you never need to hit a browser Avro, Thrift, Protobuff etc., are way better and more straightforward options to deal with than JSON.

You have a strong schema, not a retrofitted on top JSON schema spec.

The schema is enforced as the data is serialized to binary format. They have schema evolution functionality.

You get the advantages of binary format, faster de/serialization speed, less bandwidth.

You are not writing your own decoders/encoders anymore. They all have code gen to generate domain objects in your language of choice. You have the schemas in a central repo and a CI job that reads them publishing a package for your language, i.e. JAR's, NPM's, Crates etc. etc. Users just pull the artifact. This part makes dev easier. You just import the artifact and can read/write the data with no effort/code.

I've had success using Protobuff in the browser without the GRPC using protobufjs. My API is a typical HTTP Rest service, but instead of sending/receiving JSON it uses Protobuff as the message format. It has made life really easy as I just import the generated Protobuf NPM, and when I make a fetch call, I deserialize the result to an object by using the auto-generated decode function and then I can call `.getFiled()`. As I'm using Typescript on the client-side, I get type checking and type hints in the IDE. If there's no `getField` function, I know that field doesn't exist.

The advantage I now have is I don't need to do OpenAPI stuff. I get that for free. I have the request objects and response objects as protobuf so there's a strongly enforced schema file acting as the contract, and I can produce client packages in any language for free.

The other benefit is it has reduced response sizes and latency. The response size was the main reason I did a POC with protobuf as working on a mapping application and sending polylines to be displayed on a map meant I was sending 10Mb+ of JSON to the browser. Protobuff significantly reduced this.

The caveat is Protobuff are hard to debug in the browser. You can't just use the net inspector as you have binary data, not plain text.

For any personal project going forward, I'll use Protobuf instead of JSON. The biggest annoyance I have is writing JSON de/serialisers in my serverside language then having to do the same for the client-side language/app, them getting out of sync when the API evolves or making typos. Using Protobuf removes all those pain points, import the package, use the package, the specification has schematics around schema evolution so API changes is just updating to the latest package. The only downside is if you have a public API no one else does it so everyone expects JSON, a non issue for private projects.

> gRPC/protobuf has a lower operational cost than JSON as long as you don't need to talk to it from a browser.

So, reading this the other way (browsers are king)... you claim gRPC is so useless as to not be able to power your entire system and requires you to standup duplicate interchange systems for different use cases?

Yikes.

Browsers are king for some people, not others. For the stuff I'm working on, the browser is just the tip of the iceberg. For everything below the waterline, the operational benefits (strong static typing, well-defined backward- and forward-compatibility semantics, better throughput and latency characteristics) greatly outweigh the, "but we have to use a lightweight Envoy reverse proxy to expose some things to the browser," problem.

I also have a tendency to consider that Envoy proxy to be more of a feature than a bug, anyway. It's pretty easy to set up, all told. We want to gatekeep the edge, anyway, for various reasons, so it's not like there was ever a reality in which we weren't going to be fussing with a reverse proxy. And it serves as a nice opportunity to stop and be thoughtful about exactly what we're choosing to expose to the Internet.

Speaking purely as a developer, I do find it to be an annoyance. But I also acknowledge that inconveniencing developers for the sake of the greater good can be a wise move.

For a cloud-based app, that might literally just be at the application gateway. Beyond that, everything could be whatever protocol.

I would describe it as a _pyramid of bounded viability_ , from the minimally viable to the feature-burdened maximum.

CSV excels (ha) as a minimally viable exchange format for data. Combined with awk, grep, sed, bash, and Perl, and some simple SVG or D3 with SVG, the analytical solution is fast, scalable, and automatable.

But CSV has limits. The column headers are the schema. Beyond these bounds, we have the other formats.

JSON is messier. Its strength is in network/browser encapsulations and operations. As I have seen it used, people insert an array as a kind of schema, and they stay away from complex nesting where scalability starts failing and other difficult tooling must be summoned to compensate (parsing tools such as jq).

Beyond JSON's bounds, we have XML and associated tooling. XML is versatile and expressive.

XML and JSON can be written simply but both can be abused by programmers who aren't thinking beyond their own cursor.

This is a rich set of tooling for data representation.

In the end, one of the main advantages of CSV is that it remains a format that brings little tooling baggage ("ecosystem") to the task.

>CSV...has no real specification

Bite your tongue sir! It has a GLORIOUS specification!

https://datatracker.ietf.org/doc/html/rfc4180

I started reading with a light of hope I thought was long extinct, expecting this long awaited standardization of some kind, then on the first page:

> Status of This Memo

> This memo provides information for the Internet community. > It does not specify an Internet standard of any kind.

My disappointment is immeasurable and my day is ruined.

I'd argue the top of the pyramid is actually formats such as HDF5. That format was started to store voyager data and we can still read it after more than 40 years. It makes the format of entries extremely clear ("this is an 3d array of floating point numbers in IEEE755 format, with 64, 2 and 17 entries per dimension") and encourages further meat data ("it is in statV per centimeter and came out of channel of of the intrument") in addition to naming the data set "electric field". Compared to horrible piles of binary data that used to be common (and still are!), it's a breeze of fresh air to work with.

When faced with a case where SV is a natural fit, I've long been in the habit of (ass-u-ming I get to make that call, of course) specifying PostgreSQL COPY style TSV as the interchange format, and using more 'normal' TSV to make it easy to get data exports into Excel and friends.

That's turned out to be rather less annoying than any other approach to SV I've tried over the years.

> you have CSV which is popular beyond belief, and has no real specification

What about RFC 4180? Works for me.

Not work for rest of the world (called Excel)