News for hackers. Stuff that matters.
Re:Parsing gigabytes of JSON per second by cogman10 on Saturday October 23, 2021 @08:29AM (#28967781)
I've gotta say, I don't really understand why the industry is so in love with human readability for computer to computer interaction.

It seems like a notion that started in the early internet and just refuses to die.

Everything on the internet is minified and compressed at this point, so the entire idea of "human readability" left the station years ago. Yet I'll still see a primary complaint against the likes of http2+ being "it's a binary format! On NO!".

JSON seems like a similar relic. We use it not because it's fast, but because we like the idea that it's easy to decode (Even if in practice that almost never happens).

Re:Parsing gigabytes of JSON per second by secondcoming on Saturday October 23, 2021 @10:17AM (#28968761)
Indeed, out of all the trillions of HTTP requests that are communicated every day how many get read by a human. Not a lot, yet that's the use-case it's optimised for.

Those on the cloud should compute how much HTTP headers count towards their traffic egress bill.

Everyone is fine with human-readable as long as it's in English.

Binary should be the default. If writing a binary to human decoder is too much for you, you're in the wrong job.

Re:Parsing gigabytes of JSON per second by yardstick on Saturday October 23, 2021 @11:25AM (#28969382)
HTTP/2 uses a binary encoding instead of text for performance reasons. I prefer the text version myself as it’s easier to write simple tools for (quick clients, telnet based queries, etc).
Re:Parsing gigabytes of JSON per second by PeterisP on Saturday October 23, 2021 @07:11PM (#28973290)
Is a custom binary format that much more efficient than a gzipped human-readable format? If not, then it makes all sense to have a mostly universal decoder instead of a hundred different ones, especially since decoders of custom binary formats has traditionally been a major source of security vulnerabilities and other bugs even when those decoders have been written by skilled people and used in production for years.
Re:Parsing gigabytes of JSON per second by rytcio on Sunday October 24, 2021 @09:39AM (#28977779)
The answer is in your comment. If you add a step of decompression, then text parsing...it's plain-as-day going to be a lot slower then just pushing bits through a binary parser. Less work = faster
Re:Parsing gigabytes of JSON per second by AYBABTME on Saturday October 23, 2021 @08:59AM (#28967974)
It's the same idea as designing hardware for repairability. Sure you can repair something that's welded on, but it's much easier if it's bolted through instead.
Re:Parsing gigabytes of JSON per second by cogman10 on Saturday October 23, 2021 @09:30AM (#28968210)
It's not the same.

You are almost certainly passing these through tools (even built into the browser) that are doing extra processing to make it more readable.

Let's assume, for example, CBOR ends up taking over JSON. Do you not think browsers wouldn't have a CBOR parser to make it more readable?

This isn't bolt vs welding, this is bolt vs bolt with a washer. Yes there's a small extra step, but not some sort of insurmountable hurdle.

Re:Parsing gigabytes of JSON per second by pixl97 on Saturday October 23, 2021 @08:48PM (#28973838)
The questions about readability when things go wrong. When binary data breaks, mostly your just screwed in figuring out went wrong, especially when it causes your decoder to fall over. When ASCII data breaks often a view of the data can point at something even when tools can't.
Re:Parsing gigabytes of JSON per second by rytcio on Sunday October 24, 2021 @09:34AM (#28977736)
Reading a binary file is no different than reading ascii. I experimented with writing a debugger, which meant learning how to parse executable ELF binaries. It really wasn't bad at all. It just takes a bit more time to figure out what the bytes/data are expect at what offsets. The real un-said reason is a) programmers are lazy b) its easier/cheaper to hire people who can read ascii than find someone who is a bit more experienced and can figure out how to debug binary parsing.
Re:Parsing gigabytes of JSON per second by cogman10 on Sunday October 24, 2021 @11:17AM (#28978683)
Yeah and to b, it really just comes down to a lack of tools and people experienced in those tools (IMO).

There's no reason a binary format couldn't be as easy to read as a JSON format. All you'd really need is a "binary->json" gui and you're off.

It would take a little effort to make such a tool and you could have it integrated into every browser.

It's not like something like that is unprecedented even, After all, there's no browser tool out there that's showing you the gzipped resource on a compressed endpoint. It's always already doing the step of gunzipping. binary->Json would similarly be just an additional step the browser could do in the tools.

Re:Parsing gigabytes of JSON per second by djbebs on Saturday October 23, 2021 @08:58AM (#28967960)
Makes it easier to troubleshoot.
Re:Parsing gigabytes of JSON per second by NicoJuicy on Saturday October 23, 2021 @08:55AM (#28967939)
I regulary get requests from QA why something is not working. Just watching the dumped request and seeing what is wrong ( eg. Configuration) is a bless.
Re:Parsing gigabytes of JSON per second by nyanpasu64 on Saturday October 23, 2021 @05:52PM (#28972709)
A binary format you understand is easier to parse than a text format, but a binary format you don't understand is harder to learn (from reverse-engineering files) than a text format following certain conventions (HTML/JSON is more familiar to readers than PDF or Punycode). And text formats are easier to extract information out of fragmentary or corrupted documents than binary (or worse yet compressed) formats.
Re:Parsing gigabytes of JSON per second by manquer on Sunday October 24, 2021 @02:14AM (#28975368)
The human readability has to do with developer tooling and ability to work that format. Abstractions are all optimized towards developer productivity the costliest part not compute cycles.