News for hackers. Stuff that matters.
Re:MessagePack: like JSON, but fast and small by jrockway on Tuesday March 10, 2020 @03:32PM (#22538737)
JSON is a binary computer-to-computer interchange format. No computer-to-computer interchange format has comments.

The problem with JSON is that it's just human-readable enough that people think it's a config file format. It's not.

Re:MessagePack: like JSON, but fast and small by iainmerrick on Tuesday March 10, 2020 @04:18PM (#22539241)
The problem with JSON is that it's just human-readable enough that people think it's a config file format. It's not.

That attitude is misguided. Many people do in fact use JSON as a config file format, so it is a config file format. Real world usage outweighs prescriptions about what is and isn’t correct.

Looking at the first standardized JSON spec, it’s not particularly prescriptive about usage, simply saying:

JSON is syntax of braces, brackets, colons, and commas that is useful in many contexts, profiles, and applications.

Of course, it would be a better config file format if it supported comments, but people use it even despite that. I think we should try to understand the reasons for that rather than just telling people they’re doing it wrong.

Re:MessagePack: like JSON, but fast and small by foepys on Tuesday March 10, 2020 @04:23PM (#22539291)
The reason for that is that there is often a JSON parser library available for any language (especially JavaScript) but no ini or YAML config parser. So people use JSON because it's already easily accessible.

JSON is one of the worst config formats, even worse than the Apache or nginx config formats. Not being able to simply comment out a config line temporarily for testing purposes or to leave a explanatory comment for special setting is simply bad.

Re:MessagePack: like JSON, but fast and small by to11mtm on Tuesday March 10, 2020 @06:15PM (#22540404)
Having worked with a bit of YAML, RAML,(which is a superset of YAML) and JSON style configs/schemas... I'd take JSON over YAML if only because it seems to be more expressive.

Still, everyone forgets HOCON [1] is a thing, which solves many of the problems expressed here about JSON -for configuration-. It's easy to clearly specify things like time, reference other parts of the config, or if you want to change just one value, you can add that to the 'end' of the HOCON file and be GTG.

[1] - https://github.com/lightbend/config/blob/master/HOCON.md

Re:MessagePack: like JSON, but fast and small by iainmerrick on Wednesday March 11, 2020 @09:08AM (#22545483)
HOCON is meant to be a friendlier JSON, looks like? At first glance, it has some nice ideas but takes it way too far -- just the table of contents on that repo looks longer than a summary of JSON in its entirety.

Skimming through it, concatenation of unquoted values is where it goes off the rails for me. This is quite the gotcha: https://github.com/lightbend/config/blob/master/HOCON.md#not...

My wishlist for a friendlier JSON would be, in priority order:

- Allow trailing commas

- Comments

- Allow newlines instead of commas

- Allow unquoted dictionary keys

And I think that’s it. I’m not even sure the last one is worth the extra complexity.

Re:MessagePack: like JSON, but fast and small by styfle on Tuesday March 10, 2020 @09:47PM (#22542146)
I’ll have to find something wrong with HOCON and add it to my list :)

https://twitter.com/styfle/status/1237182409239658500

Re:MessagePack: like JSON, but fast and small by vimslayer on Tuesday March 10, 2020 @05:02PM (#22539642)
Though most tools that use "JSON" as the config format actually use some superset of JSON, like JSON5, and do support comments. eslintrc, babelrc, vscode config... pretty much every JS tool configuration apart from package.json.

I don't know why people would prefer that over YAML or others but at least you can actually add comments in them and the reason for their popularity doesn't seem to be baked in parser support because these tools are adding their additional "JSON" parsing anyway.

Re:MessagePack: like JSON, but fast and small by abiogenesis on Tuesday March 10, 2020 @07:34PM (#22541196)
I have a hard time remembering YAML syntax (am I constructing a list or a dict now? Oh well, better look it up) and I know for a fact that it's not just me. JSON is much simpler.
Re:MessagePack: like JSON, but fast and small by hombre_fatal on Wednesday March 11, 2020 @06:55AM (#22544696)
I even prefer JSON over TOML because JSON is simple. TOML has arbitrary rules like how `table = { foo = bar }` can't be multiline, and all that time you spend debugging your attempts at its nested table syntax, you could've just intuitively nested some dictionaries in JSON.
Re:MessagePack: like JSON, but fast and small by rumanator on Wednesday March 11, 2020 @03:44AM (#22543910)
> That attitude is misguided. Many people do in fact use JSON as a config file format

...and those people need to face the consequences of their poor and missguided technical decisions.

In this day and age we have no excuse to repeat the "but everyone is using XML for that" mistake. Just pick the right tool for the job and stop complainig that the tool needs to change to compensate for your poor judgement.

Re:MessagePack: like JSON, but fast and small by hombre_fatal on Wednesday March 11, 2020 @06:49AM (#22544679)
It's not poor and misguided. For example, VSCode has first-class support for JSONSchema giving you auto-complete and things like drop-own boxes for arbitrary project JSON config. It's better done and more mainstream than any other solution I've seen compared to people saying "well technically you could build that for <pet format>."

If you want comments, pipe it through json5 first or something. VSCode again supports comments as a courtesy. More tools that use JSON are starting to as well.

It's just not a big deal.

Re:MessagePack: like JSON, but fast and small by rumanator on Wednesday March 11, 2020 @06:55AM (#22544694)
> It's not poor and misguided. For example, VSCode has first-class support for JSONSchema giving you auto-complete and things like drop-own boxes for arbitrary project JSON config.

That's just tooling compensating for the shortcomings of a format being shoehorned into a use-case that falls outside of it's scope.

It's the XML nonsense all over again.

> It's better done and more mainstream than any other solution I've seen compared to people saying "well technically you could build that for <pet format>."

That's the same short-sighted line of argument that was used to force the mistake of using XML everywhere.

There are right tools for the right job. JSON os the right tool for a lot of jobs, but config files is not one of them.

Re:MessagePack: like JSON, but fast and small by iainmerrick on Wednesday March 11, 2020 @08:54AM (#22545365)
What are the alternatives?

Even though it’s not ideal, I actively prefer JSON over every other random format I’ve come across. This is speaking as somebody who mostly has to tweak existing configs, rather than extending them or writing new ones.

YAML (along with I guess TOML etc) looks nice at first glance, but it has too many weird syntax shortcuts that make it hard to figure out what’s actually going on. And googling for syntax like “[ ]” is hard!

With JSON, the data model is super simple, and for a given set of data there’s only one way to write it down. Sometimes the data model is too simple, sure. But even for fiddly cases like dates and times, there’s often an obvious solution (in this case, ISO 8601 strings).

Re:MessagePack: like JSON, but fast and small by Zarel on Thursday March 12, 2020 @11:32AM (#22557880)
JSON5 is a good alternative, mentioned a few times in this thread. JSON, plus comments, trailing commas, and object keys can be unquoted. Solves all my frustration with using JSON as a config file language.
Re:MessagePack: like JSON, but fast and small by iainmerrick on Thursday March 12, 2020 @03:18PM (#22560659)
Wow, I hadn’t heard of this before, even after reading other replies to this post! Looks pretty nice, hits just about every point on my personal wishlist. I’m going to use this.
Re:MessagePack: like JSON, but fast and small by gameswithgo on Tuesday March 10, 2020 @04:09PM (#22539134)
Either it is a terrible format because it is such a massively wasteful (in terms of payload size and cpu overhead to serialize and deserialize) computer to computer interchange format, or it is terrible because it doesn't have comments.
Re:MessagePack: like JSON, but fast and small by jrockway on Tuesday March 10, 2020 @04:53PM (#22539579)
Yeah, it's not a very good computer-to-computer format. No bytes type. No integer type. No Any->Any maps. Being bad at something doesn't automatically make you good at something else, and JSON is no exception. It's bad at everything except being popular.

You have to look at JSON in a historical context to understand it. It exists so that someone could get some data into their Javascript program with "eval". It was then standardized so that you didn't need a full Javascript engine to understand it, because Python and Perl and Ruby didn't have one, and it turned out that evalling random data from the Internet was a security disaster. Does that make it a good human-readable config file? Nope. Does that make it a good interface definition language? Nope.

It exists because it got popular early, and now we're stuck with it. Now it's too late to apply band-aids to make it a human-readable config language or a good computer-to-computer interchange format, because you will always have old parsers around and people will naturally want to target those. Trust me, 99% of developers will moan when you tell them that they have to use GRPC+Protos to access your API. They will be equally mad about your JSON extension that has comments and integers in it, because the dialect of Brainfuck they use for all their projects doesn't have a library that supports those extensions and their editor can't syntax-highlight it or autoformat it.

I would like to give you a solution to this problem, but there isn't one. JSON "won", but it's bad at everything except being well-understood. That seems to be all that people really care about.

Re:MessagePack: like JSON, but fast and small by mark-r on Tuesday March 10, 2020 @03:47PM (#22538887)
Any format that can be edited with a text editor and retain proper formatting should be able to have embedded comments. No reason not to, and it can come in handy sometimes.
Re:MessagePack: like JSON, but fast and small by ghayes on Tuesday March 10, 2020 @04:01PM (#22539044)
The argument, originally, I believe was that comments might be hacked to “extend” JSON with various additions (e.g. dates via an end of line marker). Comments were excluded to keep the format stable, which, FWIW, has worked.
Re:MessagePack: like JSON, but fast and small by ric2b on Wednesday March 11, 2020 @06:23AM (#22544572)
That's a bad argument, since you can still do that with "magic" fields or metadata fields.
Re:MessagePack: like JSON, but fast and small by metaloha on Tuesday March 10, 2020 @04:10PM (#22539152)
JSON isn't binary, it's text-based and human-readable by design (similar to YAML in that sense). JSONB is a binary variant, though, potentially more suitable for machine-to-machine communication and stores more efficiently in databases as a BLOB.
Re:MessagePack: like JSON, but fast and small by btilly on Tuesday March 10, 2020 @03:59PM (#22539031)
No computer-to-computer interchange format has comments.

I call BS.

As a counterexample search for the word "comment" in https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html. Or are you saying that http is not a computer-to-computer interchange format?

Re:MessagePack: like JSON, but fast and small by jrockway on Tuesday March 10, 2020 @04:58PM (#22539611)
You call BS because the HTTP/1.1 standard calls pieces of text that contain the name of the software at the other end of the connection "comments"? (To those that didn't read the spec, it uses the word comment to refer to the values in the User-Agent, Server, and Via headers.)

It turns out these "comments" were a disaster and user agents are moving away from even providing them. You put "firefox" in the user-agent header, and web servers would read the comment and send you "your browser doesn't work" instead of the actual document. That did not make for a very compatible web... turns out comments in machine-to-machine communication are a bad idea.

Re:MessagePack: like JSON, but fast and small by hoytech on Tuesday March 10, 2020 @09:35PM (#22542069)
I think the "calling BS" objection was to your categorical statement: "No computer-to-computer interchange format has comments," and not about whether they are a good idea or not (that is a separate argument).

How about ".comment" sections in ELF binaries, do they count?

https://wiki.osdev.org/ELF

Re:MessagePack: like JSON, but fast and small by btilly on Tuesday March 10, 2020 @07:11PM (#22540971)
Yes, comments tend to be turned over time into hints, directives, etc, etc, etc.

But that doesn't change the fact that people put comments into machine protocols.

For another example, lots of machine to machine protocols specify XML. Such as SOAP. They therefore all support comments.

In accord to the general trend, eventually they get abused into having a semantic meaning. For example https://support.ptc.com/help/windchill/wc111_hc/whc_en/index... shows how one system uses comments in SOAP to create documentation and a WSDL that lets interfaces to your code to be automatically generated.

And so comments eventually become executable. But that doesn't change the fact that comments can exist in machine to machine formats.

Re:MessagePack: like JSON, but fast and small by Mikhail_Edoshin on Thursday March 12, 2020 @05:55AM (#22555210)
BTW, XML actually has what can be called "executable comments": processing instructions. They are very basic, but unlike plain comments they have two parts: target name and opaque instruction for the target. If one needs to use comments for some sinister purpose, at least processing instructions may be a better fit.
Re:MessagePack: like JSON, but fast and small by yellowapple on Tuesday March 10, 2020 @05:33PM (#22539969)
XML is frequently used as a computer-to-computer interchange format, and it has comments.
Re:MessagePack: like JSON, but fast and small by didibus on Tuesday March 10, 2020 @10:56PM (#22542525)
Json? A binary computer-to-computer interchange format?

Do we use the same Json me and you?

> Json is an open standard file format, and data interchange format, that uses human-readable text to store and transmit data objects

- https://en.m.wikipedia.org/wiki/JSON

Json is text based, and meant for storage and transmission in a human readable format.

Re:MessagePack: like JSON, but fast and small by knodi123 on Tuesday March 10, 2020 @03:38PM (#22538786)
What are the material differences between a config file format, and a machine-to-machine only format?
Re:MessagePack: like JSON, but fast and small by inetknght on Tuesday March 10, 2020 @03:45PM (#22538856)
Well, for one: most config file formats specifically include a method for the user to add bytes to the data which won't be parsed or interpreted. In JSON... you might try adding a {"comment": "this is a bad way to comment"} object or key/value and hope you don't collide with an object's dedicated comment field, and also don't cause the parser to raise an error for having a field which it didn't expect.
Re:MessagePack: like JSON, but fast and small by haggy on Tuesday March 10, 2020 @03:54PM (#22538980)
Config files are typically a 'load once, read many" pattern. Machine to machine communication protocols can be VERY high rate and over the network so they need to be highly efficient both in memory footprint and serialization overhead.
Re:MessagePack: like JSON, but fast and small by iainmerrick on Tuesday March 10, 2020 @04:19PM (#22539260)
So... JSON isn’t great as a machine to machine protocol, then? So I guess it must be a config format after all!
Re:MessagePack: like JSON, but fast and small by korijn on Tuesday March 10, 2020 @03:46PM (#22538873)
The key difference is the target audience. The format's "UX", so to speak, is optimized for the target audience (machine or human). @jrockway makes an excellent point.
Re:MessagePack: like JSON, but fast and small by mark-r on Tuesday March 10, 2020 @03:50PM (#22538922)
Why must those be mutually exclusive? There's value in being understandable by both.
Re:MessagePack: like JSON, but fast and small by nitrogen on Tuesday March 10, 2020 @04:09PM (#22539142)
Having comments in a "wire format" provides a means for the spec to become meaningless as random applications start using comments to convey information (or exfiltrate data) that other applications cannot parse correctly.
Re:MessagePack: like JSON, but fast and small by mark-r on Tuesday March 10, 2020 @05:02PM (#22539653)
There should be a special place in hell reserved for people who use comments to convey machine readable information. The whole point of a comment is to provide something that has no semantic meaning.
Re:MessagePack: like JSON, but fast and small by robotnikman on Tuesday March 10, 2020 @03:57PM (#22539012)
I would think being able to have comments is a big one, so you at lease have an idea of what changes what in a config file.