News for hackers. Stuff that matters.
Re:JSON with Commas and Comments by zeroimpl on Monday February 22, 2021 @08:39PM (#26232924)
Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON parser to add optional features like skipping over comments and not caring about trailing commas. Most parsers support various flags already to configure things, so there could just be an ALLOW_COMMENTS flag and ALLOW_TRAILING_COMMAS flag.

As a case in point, the python parser breaks the JSON spec already with regards to Infinity/NaN, but then has flags to configure this. See https://docs.python.org/3/library/json.html#module-json

> The RFC does not permit the representation of infinite or NaN number values. Despite that, by default, this module accepts and outputs Infinity, -Infinity, and NaN as if they were valid JSON number literal values:

Re:JSON with Commas and Comments by k_sze on Tuesday February 23, 2021 @01:15AM (#26234326)
> Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON parser to add optional features like skipping over comments and not caring about trailing commas.

I think the most correct way to deal with this problem is to get IETF and ECMA to update the JSON standard first. Honestly, comma-after-final-element and comments are such important quality-of-life features that I never understood why they weren't part of the original spec.

On the other hand, it might be easier to just adopt TOML and forget about JSON.

Re:JSON with Commas and Comments by knome on Tuesday February 23, 2021 @01:31AM (#26234409)
>I never understood why they weren't part of the original spec

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

>Trailing commas in objects were only introduced in ECMAScript 5. As JSON is based on JavaScript's syntax prior to ES5, trailing commas are not allowed in JSON.

json was largely `eval`d when it first hit the scene. It was easy to parse because you just received it and `eval`d it directly to parse. worked in every browser. security concerns over time lead to this being rightfully replaced with the JSON object and its associated encoder and decoder.

however, since early javascript didn't allow for trailing commas, neither could JSON if it wanted to be able to be `eval`d.

Re:JSON with Commas and Comments by ritchiea on Tuesday February 23, 2021 @07:39AM (#26236595)
Interesting note: Douglas Crockford's version of JSON's origin story is great. At Yahoo Crockford wanted to return data from an API as a Javascript object because it skipped a translation step and avoided messy XML. And his bosses originally objected because there was no JSON spec. So Crockford bought a website, quickly wrote up a JSON spec and published the spec to his new official looking JSON.org site. With an official spec in place and documented on the web, there was no longer cause for his bosses to object to APIs returning JSON :)
Re:JSON with Commas and Comments by sneak on Tuesday February 23, 2021 @02:14AM (#26234630)
Comments were intentionally excluded from JSON, lest they be used to instruct the parser and cause fragmentation of the ecosystem.
Re:JSON with Commas and Comments by arcatek on Tuesday February 23, 2021 @04:27AM (#26235361)
I know it's the official reason, but it's also a really bad one - nothing prevents current JSON parsers to add some weird syntactic rules inside plain strings (similar to "use strict" in JS), but you don't see that happen. It's always been a completely hypothetical issue.

The only real reason why comments would be problematic is that they are a pain to preserve in a consistent way when editing a file, and thus would require extra work in parsers / serializers. Still, it would be worth the cost imo.

Re:JSON with Commas and Comments by sneak on Tuesday February 23, 2021 @12:34PM (#26240139)
I agree that it was stupid. I am 100% pro-comment.
Re:JSON with Commas and Comments by HelloNurse on Tuesday February 23, 2021 @06:36AM (#26236121)
In most applications reading and writing JSON are completely separate operations and roundtripping comments makes no sense.

Actual JSON-reading applications must ignore comments because they aren't allowed to care about them. On the other hand anything useful can be placed in a proper field, a comment would be a hack to pass information to human readers but not to JSON consumers (saving some time and memory). Such a technique could only be considered if people are supposed to read the JSON files and some information is useful for them but not for the reading application: a niche within a niche.

Preserving comments would actually be a complicated special purpose feature, reserved for something like structured text editors that can perform nonstandard parsing with comments included in their special object model (of the large Javascript subset/superset/variant they choose to support and roundtrip, not of JSON).

Re:JSON with Commas and Comments by ma2rten on Tuesday February 23, 2021 @01:28AM (#26234394)
I think that json is mostly a lightweight, human-readable data exchange format for machines to communicate. It's generally not meant to be written by humans.
Re:JSON with Commas and Comments by q3k on Tuesday February 23, 2021 @05:45AM (#26235785)
It's not even good for machines to communicate with, considering:

- There is no minimum/maximum number/integer value (or sizes) defined in the RFC [1] - you have no guarantee that a number you emit will be readable by all RFC-compliant parsers, so the only safe option is to emit all numbers as strings. There is also no behaviour mandated for parsers that encounter numbers/integers outside of their supported size range, so you can't rely on any fail-safe behaviour.

- There is no standardized behaviour for repeated dictionary keys (which are allowed but 'discouraged' per spec), and different implementations will treat them in different ways. This is especially painful when trying to build JSON middleware that does a parse/check/modify/emit of arbitrary data.

- Implementations in some languages (eg. Python) are non-RFC compliant by default (`python.dumps` will emit NaN/Inf/-Inf, even though the RFC forbids that), and generally all implementations are similar-but-different-enough to trip you up [2].

All three of these have bitten me in the past when trying to interface with something that spoke JSON, and as such I refuse to design new systems that use JSON as any source of truth.

[1] - https://www.rfc-editor.org/rfc/rfc8259.html

[2] - http://seriot.ch/parsing_json.php

Re:JSON with Commas and Comments by k_sze on Tuesday February 23, 2021 @01:31AM (#26234407)
There are already better formats if we just want machines to communicate though; e.g. protobuf, msgpack.

The whole reason JSON is text-based is to make it human-readable while also enabling data exchange, but the lack of comments works against that goal.

Re:JSON with Commas and Comments by BiteCode_dev on Tuesday February 23, 2021 @02:13AM (#26234624)
Not in the browser or in many stalin.
Re:JSON with Commas and Comments by oblio on Tuesday February 23, 2021 @04:43AM (#26235453)
> many stalin.

Wasn't there just one Stalin, though?

Re:JSON with Commas and Comments by bregma on Tuesday February 23, 2021 @07:00AM (#26236291)
One was certainly more than enough. Unfortunately, many have tried to pluralize.
Re:JSON with Commas and Comments by BiteCode_dev on Tuesday February 23, 2021 @07:44AM (#26236644)
:D

Stdin. New phone, forgot to disable autocorrect.

Re:JSON with Commas and Comments by matsemann on Tuesday February 23, 2021 @03:56AM (#26235192)
But it's used for all kind of config files. And there I would like to comment why stuff is done the way it is.
Re:JSON with Commas and Comments by blacktriangle on Tuesday February 23, 2021 @08:43AM (#26237107)
That's not a problem with JSON, that's a problem with people choosing JSON as a configuration file format.

I remember seeing some conversation from the JSON authors around comments and specifically not allowing comments into the spec because they did not want people to use them as extension mechanisms, so the whole no comments in JSON thing was very much intentional.

Re:JSON with Commas and Comments by dmitryminkovsky on Monday February 22, 2021 @11:06PM (#26233709)
JSON parsing is already a minefield. Please see [0], specifically this chart [1]. As mentioned in a sibling comment, I think a new mimetype makes a lot more sense sense than stirring this pot further.

[0] http://seriot.ch/parsing_json.php

[1] http://seriot.ch/json/pruned_results.png

Re:JSON with Commas and Comments by zeroimpl on Tuesday February 23, 2021 @12:37AM (#26234142)
It is a minefield, but we all walk it all the time.

A new mime type doesn’t really help because the parser doesn’t check the mime type, it assumes the programmer did that. I’m not against making a new mime type or specification, but there are already so many and making more doesn’t seem to help. Yes it would be nice if PHP and Python implement the logic for ignoring trailing commas the same way, but 99% of the time that isn’t all that important. Yes there will be cases where it matters and bugs are introduced, but since there already significant differences between JSON parsers I don’t see these kinds of things as making it much worse.

Re:JSON with Commas and Comments by spankalee on Monday February 22, 2021 @09:08PM (#26233102)
It's going to have to be a different mode and mimetype because this format would break all kinds of parsers.

I wish something would done in this area relatively soon though because JS has added a number of features that would make a new JSON much nicer like multi-line strings and BigInts.

I think JSON + comments, commas, template literals, BigInt, NaN, Infinity, and BigDecimals (if/when those land in JS) would be very useful. (It'd be nice to include dates, but that's tricky w/o a literal and because dates)

Re:JSON with Commas and Comments by fiddlerwoaroof on Monday February 22, 2021 @09:44PM (#26233302)
JSON doesn’t specify its numeric types: the mapping of a string of digits to concrete numeric types is implementation-defined: so, JSON doesn’t need specific syntax for BigInts or arbitrary-precision decimals.
Re:JSON with Commas and Comments by spankalee on Monday February 22, 2021 @10:15PM (#26233475)
Current parsers cannot start returning BigInts instead of numbers without that being a breaking change. And I'm not sure anyone wants a format where the result may change types based on the size of the number, especially for languages where arbitrary precision types are not compatible with other numbers.
Re:JSON with Commas and Comments by skissane on Monday February 22, 2021 @10:34PM (#26233563)
> Current parsers cannot start returning BigInts instead of numbers without that being a breaking change.

Current parsers aren't uniform here. Since the JSON spec is silent on what post-parsing format is used for numbers, each parser is free to do whatever makes sense in the context of the host language. I reckon you'll find some JSON parsers use bigints already, especially in languages with first-class bigint support (such as various Lisp dialects)

> And I'm not sure anyone wants a format where the result may change types based on the size of the number

That's nothing to do with the format, that's to do with the parser. Some parsers already do exactly that – use an integer type for numbers that are integers, use a floating point type for numbers that contain decimal points

Re:JSON with Commas and Comments by banana_giraffe on Monday February 22, 2021 @11:13PM (#26233744)
> I reckon you'll find some JSON parsers already use bigints already

Yep. Python is one such language. I've seen this catch people by surprise when they discover their serial number (which granted, should have been a string in the first place) doesn't survive a trip from Python to JSON to Javascript, among other languages.

Re:JSON with Commas and Comments by colejohnson66 on Tuesday February 23, 2021 @08:31PM (#26245511)
It doesn’t help that it’s called a serial number, and is sequential (hence, serial, but some manufacturers don’t care). num++ is easier than implementing increment_number_string that works on ASCII digits.
Re:JSON with Commas and Comments by banana_giraffe on Tuesday February 23, 2021 @08:47PM (#26245610)
Yep, "serial number" was basically the argument for using a number type in the first place, and here they were sequential. Even then, ignoring the JSON peculiarities, it would have been ok up till someone had the brilliant idea to have x digit serial numbers for one product line, and y digit for another. With only zero padding to tell them apart. This person, not a programmer, could not fathom why it caused the "tech heads to go crazy".

Sometimes I debate collecting stories like this.

Re:JSON with Commas and Comments by xpe on Monday February 22, 2021 @09:07PM (#26233101)
I can see why this example appeals to you. However, asking implementors to go outside of a spec is _in effect_ making a new JSON variant. Once you see this, you can understand why people want a spec with a name.
Re:JSON with Commas and Comments by bshimmin on Tuesday February 23, 2021 @06:46AM (#26236204)
Wouldn't you just end up with (for example) "Python-flavoured JSON", just like we have "GitHub-flavoured Markdown"?

On the other hand, there is an RFC for JSON where there isn't for Markdown, so it ought to be possible to make use of the usual standards process to formalise a successor to the original JSON which might include some of the very common additions some people always want (trailing commas, comments, proper dates) - that this hasn't happened over the last 15 years suggests really that the problem is actually due to lack of agreement.

Re:JSON with Commas and Comments by lifthrasiir on Tuesday February 23, 2021 @01:44AM (#26234476)
Seriously, I believe the omission of Infinity and NaN from JSON is a huge mistake, if not Crockford's bad joke. It is commonly said that JSON was originally going to be `eval`ed and Infinity and NaN could have been redefined, but that eval had to be already preceded by filtering anyway so you can put a few lines to ensure that Infinity and NaN are expected values. Or use `Function` instead. Or use `1/0` or `0/0` as pseudo-literals. Not every JS value is present in the JSON data model, but it's absurd that not every JS number is present in it.
Re:JSON with Commas and Comments by acje on Tuesday February 23, 2021 @01:36AM (#26234434)
Or just do RSON? https://github.com/rson-rs/rson

I honestly don't want to bash JS, but typing is not it's strength.

(Not affiliated)

Re:JSON with Commas and Comments by saghm on Tuesday February 23, 2021 @01:43AM (#26234471)
I think that's the opposite of what GP is advocating; they're arguing that a better approach than defining entire new formats, we should just augment existing JSON implementations. I'd guess that the rationale is that it would be a tougher uphill battle to get people to switch to an entirely new format, but adding new features to implementations they're already using wouldn't require making anyone switch.
Re:JSON with Commas and Comments by acje on Tuesday February 23, 2021 @02:43AM (#26234782)
And I generally think good constraints are more empowering than good options when it comes to any form of systems design. It took me 20 years in the industry and 20 minutes with cynefin to understand this.
Re:JSON with Commas and Comments by marcosdumay on Tuesday February 23, 2021 @02:07PM (#26241443)
While we are at it, how about making the parsers accept non-ascii space, so that things don't break if somebody using Windows edit a file somewhere and adds a BOM.
Re:JSON with Commas and Comments by thayne on Tuesday February 23, 2021 @12:56AM (#26234233)
ruby's JSON.parse accepts comments (but not trailing commas)