News for hackers. Stuff that matters.
Re:The Pretty JSON Revolution by Garlef on Tuesday February 23, 2021 @10:58AM (#26238779)
> JSON can be made prettier by sorting the JSON object members by element keys.

This seems to be a bad idea. The JSON language spec has ORDERED object members. But the order is arbitrary (precisely the one given in the JSON string) and does not have to be the lexicographic.

Sorting the object members by default would introduce problems whenever the order matters to the consumer of the JSON.

Re:The Pretty JSON Revolution by dragonwriter on Tuesday February 23, 2021 @12:46PM (#26240298)
> The JSON language spec has ORDERED object members.

False. “An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.” [emphasis added][0]

[0] https://tools.ietf.org/html/rfc8259

Re:The Pretty JSON Revolution by ucarion on Tuesday February 23, 2021 @01:22PM (#26240720)
Not exactly, unfortunately. The text you cite is in the introduction, which is non-normative. That text talks about the conceptual data model, but that's just to frame the reader's thinking.

The normative text has the "real" answer, and the real answer is that it's basically undefined behavior. It starts by saying "The names within an object SHOULD be unique", and then elaborates:

  An object whose names are all unique is interoperable in the sense that all
  software implementations receiving that object will agree on the name-value
  mappings.  When the names within an object are not unique, the behavior of
  software that receives such an object is unpredictable.  Many implementations
  report the last name/value pair only.  Other implementations report an error
  or fail to parse the object, and some implementations report all of the
  name/value pairs, including duplicates.
  
  JSON parsing libraries have been observed to differ as to whether or not they
  make the ordering of object members visible to calling software.
  Implementations whose behavior does not depend on member ordering will be
  interoperable in the sense that they will not be affected by these
  differences.
https://tools.ietf.org/html/rfc8259#section-4
Re:The Pretty JSON Revolution by dragonwriter on Tuesday February 23, 2021 @01:54PM (#26241225)
> The normative text

Unlike some RFCs, which clearly and explicitly delineate normative from informative material, RFC 8259 does not, but the text you cite is on its face informative rather than normative: it does not specify what an implementation MUST or SHOULD do, or what the object model IS, it describes the variety of preexisting implementations (based, correctly or not, on prior specifications) that are in the wild.

Re:The Pretty JSON Revolution by hyperion2010 on Tuesday February 23, 2021 @01:37PM (#26240961)
This reads like multiple instances of the same name in a json object is undefined behavior and the user will get what they deserve if they try to rely on that behavior.
Re:The Pretty JSON Revolution by gpvos on Tuesday February 23, 2021 @06:03PM (#26244208)
So basically, "it's a free-for-all, we guarantee absolutely nothing unless you 1) don't generate duplicate keys and 2) the code of your consumer doesn't depend on their ordering".
Re:The Pretty JSON Revolution by ucarion on Wednesday February 24, 2021 @04:48PM (#26256506)
Yeah. Maybe it was a bit unclear, but when I used the term "undefined behavior" I think I meant exactly what "free-for-all" means in your sentence.
Re:The Pretty JSON Revolution by msluyter on Tuesday February 23, 2021 @11:29AM (#26239198)
In theory, order shouldn't matter, right? But I recall seeing this trick for adding comments to json:

  { 
    "foo": "this is a comment about foo",
    "foo": "actual value of foo that overwrites the comment"
  }
The trick is that the second value value of foo overwrites the first. But, clearly, sorting would would wreak havoc here (if the value was used in the sort key). ;)
Re:The Pretty JSON Revolution by Buttons840 on Tuesday February 23, 2021 @12:39PM (#26240205)
Yes. That is valid JSON.

A fun fact about MongoDB is it will actually store that JSON, both duplicate keys. The implication is that whatever MongoDB client you're using, that maps Mongo data to dictionaries/maps, is not capable of representing all valid MongoDB documents. It's important to recognize that Mongo may be storing data your client will not be able to access.

I learned this when the Python client was showing one value for a key, and the Ruby client was showing another value for the same key, and neither client was showing the whole document.

Re:The Pretty JSON Revolution by jd_mongodb on Wednesday February 24, 2021 @08:37AM (#26249674)
I can't reproduce that behaviour on MongoDB 4.2.x using Python and the Shell client.
Re:The Pretty JSON Revolution by Buttons840 on Wednesday February 24, 2021 @04:10PM (#26255994)
It was the case some years ago. I'm not sure if it still is the case.
Re:The Pretty JSON Revolution by Latty on Tuesday February 23, 2021 @11:13AM (#26239001)
The spec[1] says:

> An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.

"whenever the order matters to the consumer of the JSON" should be never.

More pragmatically, regardless of what the spec says, a ton of JSON tooling assumes the order doesn't matter and relying on it would be a big mistake.

[1]: https://tools.ietf.org/html/rfc7159#section-1

Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @11:17AM (#26239051)
The other spec (ECMA-404) disagrees with RFC 7159 on this point.

That's the great thing about specs -- if you don't like what one says, there's always another to support your position. :)

Re:The Pretty JSON Revolution by Latty on Tuesday February 23, 2021 @11:24AM (#26239145)
Well, falling back to the pragmatist position: a lot of JSON tooling assumes the order isn't relevant semantically. Not taking that position will cause you a lot of headaches.
Re:The Pretty JSON Revolution by stickfigure on Tuesday February 23, 2021 @01:09PM (#26240543)
...and a lot of other tooling assumes the order is relevant. Quite a lot of application logic in the real world, too. Which is why browsers preserve iteration order.

JSON is a serialization format. Its components inherently have a serial order. You can't change this any more than you can legislate the value of pi to be 3.

Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @01:23PM (#26240729)
Nitpicking, but JSON was designed as a data-interchange format, it only happens to be used for serialization. I agree that any kind of pipeline component (i.e., not a source, and not a sink) should preserve ordering where possible, for the sake of robustness.
Re:The Pretty JSON Revolution by stickfigure on Tuesday February 23, 2021 @02:49PM (#26241984)
JSON is defined (and has always been) as a serial stream of characters. I think it's fair to call that serialization.
Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @03:08PM (#26242232)
I'm not disagreeing that JSON is used for data serialization, but your last comment just muddies the waters. There are plenty of things that are serial in nature, but have nothing to do with "serialization" in the sense of data marshalling. I guess that the term is kind of unfortunate.

Consider just how how many data formats are ultimately defined as a "serial stream of characters" -- and then consider how few of those you would practically use for marshalling a general data structure.

Re:The Pretty JSON Revolution by mywittyname on Tuesday February 23, 2021 @01:54PM (#26241224)
Well, you, as a programmer, can make the assumption that anyone creating JSON for consumption by your program will obey this requirement. Generally speaking, it's not a good idea to make that assumption, because even people within your team or company may not know of that rule or expect it to be enforced and may break it.

Also, there's the problem of different JSON libraries behaving differently. Such as using unordered hashmaps as an internal data representation for parsed content, making compliance difficult.

Re:The Pretty JSON Revolution by Latty on Wednesday February 24, 2021 @07:37AM (#26249231)
Sure, but we were talking about being a consumer. In general, I would argue Postel's law: be careful with what you send, be liberal with what you accept.

Don't mess with the order yourself, but don't assume other tooling will respect it.

Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @12:19PM (#26239922)
Agreed, and if you want to write tools that are conformant with both specs, you really don't have a choice. The RFC pretty much spells out this point: "JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences."

(This kind of text is why I prefer the ECMA document -- it's clearly written to be a normative standard, rather than as a field-report and Request for Comments.)

Re:The Pretty JSON Revolution by dragonwriter on Tuesday February 23, 2021 @01:02PM (#26240474)
That ECMA (via ECMA-404) and IETF (via RFC 8259, and before that RFC 7159) have subtly incompatible standards with the same title is annoying; perhaps we need to talk about “IETF JSON” (or “application/json”, as it is expressly the basis of the MIME type) vs “ECMA JSON”.
Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @03:35PM (#26242531)
Here is our opportunity to publish an ANSI or ISO spec, and take the market by storm. :)

Personally, I will adopt the first version of JSON that lets me insert a flipping comment!

Re:The Pretty JSON Revolution by Garlef on Tuesday February 23, 2021 @11:25AM (#26239148)
> More pragmatically, regardless of what the spec says, a ton of JSON tooling assumes the order doesn't matter and relying on it would be a big mistake.

Agreed. But this does not mean that a tool should break it.

My assumption would be that

   fn(parse(pretty_print(someJSONString)))
should always evaluate to the same as

   fn(someJSONString)
(for all functions fn)
Re:The Pretty JSON Revolution by fiddlerwoaroof on Tuesday February 23, 2021 @12:53PM (#26240371)
JavaScript objects are ordered maps (as are PHP’s arrays, iirc): it’s not common, but relying on this is specified behavior in JavaScript: it would be a little surprising for JSON object literals to follow a different rule.
Re:The Pretty JSON Revolution by dragonwriter on Tuesday February 23, 2021 @01:15PM (#26240626)
> JavaScript objects are ordered maps (as are PHP’s arrays, iirc): it’s not common, but relying on this is specified behavior in JavaScript: it would be a little surprising for JSON object literals to follow a different rule.

JSON object literals expressly follow a different rule (are expliclty unordered) per the IETF specs and have no specific significance to order at the JSON level though some might conceivably be introduced in ancillary specifications or tooling per the bigECMA spec.

JSON is syntactically a subset of JS but not semantically identical. Unambiguous order would require an array of one-entry objects in JSON.

Re:The Pretty JSON Revolution by fiddlerwoaroof on Tuesday February 23, 2021 @01:29PM (#26240831)
This isn’t true, the ECMA spec for JSON specifies that order is treated in an implementation-defined fashion.

> An object structure is represented as a pair of curly bracket tokens surrounding zero or more name/value pairs. A name is a string. A single colon token follows each name, separating the name from the value. A single comma token separates a value from a following name. The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.

https://www.ecma-international.org/wp-content/uploads/ECMA-4...

This wording allows a particular implementation to define its own meaning to the order of key-value pairs, or even to produce a multimap.

Re:The Pretty JSON Revolution by Garlef on Tuesday February 23, 2021 @11:10AM (#26238952)
I checked the spec again:

It's unclear: At one point it says "An object is an unordered set of name/value pairs." while in the actual grammar it is ordered:

    object
        '{' ws '}'
        '{' members '}'
    
    members
        member
        member ',' members
Re:The Pretty JSON Revolution by gmfawcett on Tuesday February 23, 2021 @11:14AM (#26239005)
I recommend reading the ECMA-404 spec instead. It's less ambiguous, and basically says that you can treat the pairs as ordered if you want, as the syntax itself doesn't imbue the order with any meaning:

https://www.ecma-international.org/wp-content/uploads/ECMA-4...

Re:The Pretty JSON Revolution by dragonwriter on Tuesday February 23, 2021 @01:17PM (#26240645)
Grammar is inherently ordered, semantics is different from syntax.
Re:The Pretty JSON Revolution by peterohler on Tuesday February 23, 2021 @11:13AM (#26238995)
I think you will find the JSON object members are not ordered while JSON array members are ordered. Since the JSON object members are not ordered, changing the order for display purposes does not change the data in any material way.