News for hackers. Stuff that matters.
Re:JSONB Has Landed in SQLite by cryptonector on Wednesday December 06, 2023 @11:31AM (#38546075)
JSON processors are not required nor expected to retain object key ordering from either input or object construction order.
Re:JSONB Has Landed in SQLite by fl0ki on Wednesday December 06, 2023 @12:38PM (#38547048)
That compounds really badly with another lack of specificity in JSON: parsers can treat duplicate keys in arbitrary ways, e.g. keep only the first or last value, combine values somehow, error out, etc.

When keys are not unique, their order can matter again, and can matter differently to different parsers.

Re:JSONB Has Landed in SQLite by btown on Wednesday December 06, 2023 @12:42PM (#38547103)
The question is whether people think of their database as a processor of data or a storage location for data, exactly as it was provided. Given that a lot of people use SQLite as the latter, it's a worthwhile caveat to make people aware of.

Additionally, Javascript environments (formalized in ES2015), your text editor, and your filesystem all guarantee that they'll preserve object key order when JSON is evaluated. It's not unreasonable that someone would expect this of their database!

Re:JSONB Has Landed in SQLite by cryptonector on Wednesday December 06, 2023 @06:26PM (#38550896)
> The question is whether people think of their database as a processor of data or a storage location for data

If you use the database's JSON type(s) and/or functions, then yes, your database is a JSON processor.

And, yes, your database is not a dumb store, not if it's an RDBMS. The whole point of it is that it's not a dumb store.

Re:JSONB Has Landed in SQLite by seanhunter on Wednesday December 06, 2023 @02:14PM (#38548232)
If you're storing it exactly as provided then surely until you know it's actually valid json you want a string, not a json/jsonb column.
Re:JSONB Has Landed in SQLite by ash on Wednesday December 06, 2023 @12:50PM (#38547211)
> Javascript environments ... guarantee that they'll preserve object key order when JSON is evaluated

Yes, but only for non-integer keys.

Re:JSONB Has Landed in SQLite by gmfawcett on Thursday December 07, 2023 @06:20PM (#38563175)
Is there such a thing as a non-string JSON key to be evaluated? https://www.json.org/ suggests not.
Re:JSONB Has Landed in SQLite by ash on Thursday December 14, 2023 @09:30AM (#38641692)
Yes, in the context of JSON handling by JavaScript. I meant string keys that can be interpreted as non-negative numbers. Those are traversed first, the rest of the keys are coming in insertion order. For example:

    > JSON.stringify({'one': 1, '3': 3, 'two': 2, '2': 2, 'three': 3, '1': 1})
    '{"1":1,"2":2,"3":3,"one":1,"two":2,"three":3}'
Re:JSONB Has Landed in SQLite by gmfawcett on Sunday December 17, 2023 @09:42AM (#38673157)
Thanks for explaining. That behaviour is... interesting!
Re:JSONB Has Landed in SQLite by ash on Wednesday December 06, 2023 @12:48PM (#38547177)
Yes, but it's not impossible to achieve it in practice.

For example, JavaScript JSON.stringify and JSON.parse have well-defined behaviour:

> Properties are visited using the same algorithm as Object.keys(), which has a well-defined order and is stable across implementations

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

> The traversal order, as of modern ECMAScript specification, is well-defined and consistent across implementations. Within each component of the prototype chain, all non-negative integer keys (those that can be array indices) will be traversed first in ascending order by value, then other string keys in ascending chronological order of property creation.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Similarly, Python json module guarantees that "encoders and decoders preserve input and output order by default. Order is only lost if the underlying containers are unordered." Since 3.7 dict maintains insertion order.

https://docs.python.org/3/library/json.html

https://docs.python.org/3/library/stdtypes.html#dict

Yes, JS and Python behaviour are not the same for all cases, however, non-integer keys do maintain order across Python and JavaScript.

Re:JSONB Has Landed in SQLite by cryptonector on Wednesday December 06, 2023 @01:00PM (#38547337)
> Yes, but it's not impossible to achieve it in practice.

For any one implementation. But there's a very large number of implementations. You just can't count on object key order being preserved, so don't.

Re:JSONB Has Landed in SQLite by anamexis on Wednesday December 06, 2023 @01:50PM (#38547970)
You can count on object key order being preserved when you control the implementation. Which is why it's useful to know whether a given implementation preserves key order.
Re:JSONB Has Landed in SQLite by ash on Wednesday December 06, 2023 @01:24PM (#38547665)
There are situations where keeping order is useful. For example, human-editable json config files.

I also gave an example of two implementations that are compatible for a useful subset of keys. By the way, SQLite JSONB keeps object keys in insertion order, similar to Python: https://news.ycombinator.com/item?id=38547254

Re:JSONB Has Landed in SQLite by cryptonector on Wednesday December 06, 2023 @06:28PM (#38550908)
> There are situations where keeping order is useful. For example, human-editable json config files.

You might have to just normalize every time you want to edit that.