But I think complexity is always 90% culture. It's pretty arbitrary what kind of culture grows around a particular technology.

I.e. learning about namespaces would take a programmer couple of hours, including a foosball match and a coffee break, but working around JSONs bad decisions when it comes to number serialization or sequence serialization will probably take days in the best case, with a side-effect that this work will most likely have to be done on an existing product after a customer complained about corrupting or losing their data...

some potential issues https://bishopfox.com/blog/json-interoperability-vulnerabili...

on edit: not parent commenter of course, just what I think they might have meant.

So, for a while there weren't too many objects, ids appeared to be all different... until they weren't. It's easy to see how this led to data corruption, I suppose?

---

Here's a hypothetical example: few people are aware that JSON allows key duplication in "hash-tables", also, even if they consider such a possibility they might not know that JSON doesn't prescribe which key should win, should there be many of them. They might assume that the definition requires that the first chronologically wins, or last, or... maybe some other rule, but they hope that it's going to be consistent across implementations.

Obviously, to screw with developers, JSON doesn't define this. So, it's possible that two different parsers will parse the same JSON with the same fields differently. Where this could theoretically explode? -- Well, some sort of authentication which sends password with other data that can be added by user, and the user intentionally or accidentally adds a "password" field, which may or may not later be overriden and may or may not later be interpreted on the other end as an actual password.

---

There are many other things, like, for example, JSON has too many of the "false" values. When different languages generate JSON they may interpret things like "missing key" and "key with the value null" as the same thing or as a different thing. Similarly, for some "false" and "null" are the same thing, while for others it's not.

I would say it's the other way around. Many people seem to think that duplicate keys are allowed in JSON, but the spec says "An object is an unordered set of name/value pairs". Sets, by definition, do not allow duplicates.

https://www.json.org/json-en.html

>There are many other things, like, for example, JSON has too many of the "false" values. When different languages generate JSON they may interpret things like "missing key" and "key with the value null" as the same thing or as a different thing. Similarly, for some "false" and "null" are the same thing, while for others it's not.

I don't see how this is a JSON issue. There's only one false value in JSON. If some application code or mapping library is hellbent on misinterpreting all sorts of things as false then there is no way to stop that on a data format level.

What I do agree with is your critcism of how the interpretation of long numbers is left unspecified in the JSON spec. This is just sloppy and should be fixed.

It's not about the time it takes to learn about namespaces. I'm talking about the complexity that namespaces and entities add to the data model and the requirement to actually handle them throughout the entire stack.

You can normalise and compare arbitrary pieces of JSON using only information available locally in that same sequence of UTF-8 bytes. You cannot do that with XML. You have to consider the whole document context and resolve all namespaces and entities before actually comparing anything.

The JSON specification is ~5 pages and most of that is diagrams. The XML specification is ~40 pages long and it imports ~60 pages of URI specification.

I'm not saying that it's impossible to use only the simple parts of XML unless and until you actually need what namespaces have to offer. But that's culture, and you have no control over other people's culture.

I've worked a lot with XML, and I have no idea what complexity are you talking about. This just wasn't complex / difficult. Once you've learned what this was about, this was your second nature. Eg. I spent a lot of time working with MXML -- that is an XML format for Adobe Flex markup similar to XAML and a bunch of others of the same kind. It used XML namespaces a lot. But that was the least of my problems using it...

Again, I've never had anyone who learned how and why to use XML namespaces complain about it. All complaints about this feature were coming from people discovering it for the first time.

> You can normalise and compare arbitrary pieces of JSON

Dream on. No, you cannot. It depends on parser implementation. For example, you have two 20-digit numbers where 15 most significant digits are the same. Are these numbers the same number or a different number in JSON?

The fact that it's 5 pages means nothing... it's 5 pages that define a bad language that creates a lot of problems when used. So what if it only took 5 pages to write it? You can probably squeeze Brainfuck definition into half a page? -- So what, it's still a lot harder to use than JavaScript.

>Dream on. No, you cannot. It depends on parser implementation. For example, you have two 20-digit numbers where 15 most significant digits are the same. Are these numbers the same number or a different number in JSON?

That's just a mildly interesting interoperability edge case that can be worked around. I agree that it's not good, but it is a problem on a wholly different level. XML elements not being comparable without non-local information is not an edge case and not an oversight that can be fixed or worked around. It's by design.

I'm not criticising XML for being what it is. XML tries to solve problems that JSON doesn't try to solve. But in order to do that, it had to introduce complexity that many people now reject.

Edit: I think we're talking past each other here. You are rightly criticising the JSON specification for being sloppy and incomplete. I don't dispute that. I'm comparing the models as they are _intended_ to work. And that's where XML is more complex because it tries to do more.

"XML Namespaces: Giving developers the vapors since 1999."

The XML specification does have Document Type Definitions (DTD), which were sort of inherited from SGML. This is an optional declaration with its own syntax that defines a DTD. I don't think they were that widely used. XMl Schema started out as an attempt to redefine those in XML.

The nice thing with XML Schema was that you could usually ignore them and just use them as documentation of stuff that you might find in a document. Typically, schema urls wouldn't even resolve and throw a 404 instead. More often than not actually. My go-to tool was xpath in those days. Just ignore the schema and cherry pick what comes back using xpath. Usually not that hard.

The culture around Json is that it emerged out of dynamic language communities (Javascript, Ruby, Python, etc.) with a long tradition of not annotating things with types and a natural aversion against using schemas when they are not needed. Also, they had the benefit of hindsight and weren't looking to rebuild the web services specs on top of json but were actively trying to get away from that.

I know, and I'm not talking about XML Schema at all (partly because it hurts my brain to even mention the absolute worst specification ever written).

I mean just the complexity of the XML data model itself, including namespaces, entity references and the ridiculously convoluted URI spec. That's more than enough to make XML far more complex than JSON.

To be fair, XML solves problems that JSON doesn't solve. JSON is not a better XML. JSON's creators simply decided that many of problems that XML solves don't need solving or should not be solved by a data format specification.

    <some>true</some>

    <some>1</some>

For example, MS claims that for exchange ASD boolean values must be integer 1 or 0 [0], but then links to a W3C spec that allows for the tokens true and false [1]

At least with JSON and HTML, you don't need a separate definition file for basic, primitive data types.

[0] https://learn.microsoft.com/en-us/openspecs/exchange_server_...

[1] https://www.w3.org/TR/2004/REC-xmlschema-2-20041028/#boolean

This would've been useful if you knew what kind of number it was...

As for what goes into separate elements and what goes into attributes: a typical answer to this is that simple types (as per XSL) go into attributes, complex types go into elements.

Compare this to JSON's screwed-up definition of "hash-tables" (the things in curly braces) which doesn't require that "keys" be unique.

XML wasn't perfect. But JSON isn't really better. It sucks in a slightly different way because people keep inventing these formats w/o much thinking, and once discover problems, don't fix them.

JSON isn’t ambiguous when it comes to this. Numbers are arbitrary precision decimal numbers[1].

I’m guessing your issue is with how Javascript interprets JSON numbers as 32 bit floats. But that is a (mis-)feature of JavaScript and switching your serialization format to XML would not help, because JavaScript represents all numbers as 32-bit floats.

[1] https://www.json.org/json-en.html

JSON on the other hand does not separate types and data, the types are implicitly contained in the data. So you can get type information from the data without a schema, at least up to the point where JSON's simple type system is no longer expressive enough, then you need - just as with XML - a schema to get the correct type information, for example to distinguish actual strings and dates.

If you really need this, nobody stops you from including type information in XMLs, have a type attribute on elements or <quote>"strings"</quote> as compared to <numbers>123</numbers> and use that. You will of course have to do this on your own, that is just not the way XML is supposed to be used.

Which, in practice, is a terrible oversight. I've honestly never seen a JSON store/transport/serde in practice without dates and/or times in them. There's always some updated_at or captured_on somewhere in the API or dataset.

Of all the data-types needed, I'd say dates are amongst the most important. At least more important in practice than floats; which JSON does support for odd reasons. Especially with dates being ambigous at best and inconsistent at worst.

Pubquiz: when was/or will be, how much paid? { currency: "THB", paid_at: "04-03-2566", amount: 13.37 }¹ - JSON is neat for simple use-cases, but utterly impractical for when precision and correctness is required. Yet here we are, building around and on top of it, to get that correctness and precision.

¹I'm messing a bit, 'cause this calendar isn't used in practice for such use-cases anymore. Hardly. But I've seen this with Hijri calendars. And those silly US date-formats. I've seen it "solved" with complex structs like { created_at: { year: 2022, day: ... , timezone:xxx}". I've myself "fixed" floating-point precision issues in financial applications that used JSON by using all-strings: "{ currency: "USD", amount_in_cents: "1337" } and such.

There's standard to encode dates to string. It's called ISO-8601 and it's supported everywhere.

Also JSON poses no particular limits which would force anyone to encode string as number.

No. I'm complaining that JSON is too limited. And that it's "type system" is lacking so much that I have to resort to hacks like encoding numbers in strings. Which I think is embarrassing for an industry.

> There's standard to encode dates to string. It's called ISO-8601 and it's supported everywhere.

It's not. Too many servers and services use formats other than ISO-8601. Should I call Visa that their export formats suck? Or that Random API that their JSON datefields should be changed to ISO-8601. It's supported in most languages. But e.g. something widely used as Google Sheets doesn't support this: If you get a JSON or CSV with ISO-8601 into Google sheets, a lot of string parsing and even regexes are needed to turn it into a proper date.

Saying "we use ISO-8601 and that solved everything" only works if you never need any service outside of yours and never interop or exchange data with other services. Which in practice is never for anything remotely successful.

I have yet to see Time be easy. Anywhere. At all. From daylight savings being state-dependent, system times resetting to rand, right down to CPU monotonic timing.

Using Time handling as a criticism to JSON's architecture doesn't hold water.

You mean a float, right?

But JSON doesn't disallow arbitrary precision numbers, that's up to the parser implementation.

   number
    integer fraction exponent

That's exactly what I'm doing. And indeed another reason why I feel embarrassed by JSON. I mean, we -the industry- have been doing financial data transport over computer networks, for how long now? fifty years? And we keep "inventing" transport formats that unsolve issues that have long been solved and done. XML had this solved[1]. Hell, even the ancient MT940[2] had this solved.

[1] https://web.archive.org/web/20200618100100/https://deutscheb... (pdf warning) [2] e.g. https://financialdataexchange.org/FDX/About/OFX-Work-Group.a...

> At least with JSON and HTML, you don't need a separate definition file for basic, primitive data types.

Unless I’m missing your meaning, this seems like an apples-to-oranges comparison. HTML is not a general-purpose format like JSON. It’s a very complicated document format that is validated with reference to an external spec.

I think XML is a great fit for a document format that can become arbitrarily complex yet still easy to author and validate. It’s obviously a really poor fit for a wire transport protocol.

unless you need something different from JavaScript primitive data types.

For example integers.

Or null means nothing to you.

Or you want a faithful representation of input

   Welcome to Node.js v20.5.1.
   Type ".help" for more information.
   > JSON.stringify(undefined)
   undefined

   > JSON.stringify([undefined])
   '[null]'

   jq "." <<< "[null]"     
   [
     null
   ]
   
   jq "." <<< "undefined"
   parse error: Invalid numeric literal at line 2, column 0

`JSON.stringify()` is documented to behave exactly as you demonstrate, so there's no surprises:

undefined, Function, and Symbol values are not valid JSON values. If any such values are encountered during conversion, they are either omitted (when found in an object) or changed to null (when found in an array). JSON.stringify() can return undefined when passing in "pure" values like JSON.stringify(() => {}) or JSON.stringify(undefined).

Expecting `jq` to somehow understand that its input came from Javascript's `JSON.stringify()` and so should be parsed on that basis seems ... odd? I don't see any problem with what `jq` is doing there, but anyway I don't see a problem with JSON itself in these examples.

that's the point.

The official JSON serializer from every broswer vendor and every Node installation produce invalid JSON.

Which for the JavaScriptObjectNotation is kinda hilarious.

> Expecting `jq` to somehow understand that its input came from Javascript's `JSON.stringify()`

I would expect `JSON.stringify` to give an error if trying to serialize something that naturally does not map to JSON, like many other libraries do.

You have to provide a manual override for those situations.

But JavaScript and ECMA (`JSON.stringify` is defined in the standard) decided that no, they can ignore the specs for some reason.

Problem is they can't fix it now, because too many applications rely on those wrong assumptions.

Here it is the reason why you can find <flag>true</flag> and <flag>1</flag>

Difference being XML was born to standardize the document format, JSON aspired to be a data format but failed miserably at it, even at the most basic level, like saying an int from a float. The spec is simply too vague and ambiguous to give some guarantee of interoperability, beyond numbers and strings.

Maybe we should all switch to MessagePack

    > JSON.stringify(42)
    '42'
    > JSON.stringify(undefined)
    undefined
    > typeof JSON.stringify(42)
    'string'
    > typeof JSON.stringify(undefined)
    'undefined'

I don't think I agree with this. `JSON.stringify` isn't producing JSON when it returns `undefined`. Instead...

> I would expect `JSON.stringify` to give an error if trying to serialize something that naturally does not map to JSON, like many other libraries do.

> You have to provide a manual override for those situations.

... `undefined` is an error. As in, there's no meaningful difference between catching an exception and providing "a manual override" for `undefined`, is there?

> But JavaScript and ECMA (`JSON.stringify` is defined in the standard) decided that no, they can ignore the specs for some reason.

What part of what spec is being ignored? `JSON.stringify` conforms to its own spec, as you say; and when it returns JSON, the JSON is valid. Meanwhile the JSON spec itself is very explicit about not declaring rules for serialisation/deserialisation:

The goal of this specification is only to define the syntax of valid JSON texts. Its intent is not to provide any semantics or interpretation of text conforming to that syntax. It also intentionally does not define how a valid JSON text might be internalized into the data structures of a programming language. There are many possible semantics that could be applied to the JSON syntax and many ways that a JSON text can be processed or mapped by a programming language. Meaningful interchange of information using JSON requires agreement among the involved parties on the specific semantics to be applied. Defining specific semantic interpretations of JSON is potentially a topic for other specifications.

XML is a language for marking up text. SVG uses attributes for all vector data, because the vector points are not meant to be presented to a user as raw data.

If I embed a SVG into a XHTML document and the browser does not understand SVG, the text within the graphic is still presented to the user.

> Even worse, nothing is sanely typed without an xsd. Different systems will treat the following differently:

This is not a responsibility of XML, which deals in a common well-formed markup format for various document format.

It sounds like you are dealing with a tool that has defined an XML-based data interchange format, and that they may have inconsistent tooling for their format.

I think a good rule of thumb is that attributes are for key/value pairs that are probably not user-visible and definitely not directly user-editable.

Carried to a logical conclusion, this would simplify the auto-creation of form GUIs.

Are you ok with <a href="..">link</a>?

That was kind of my original point, people are fine with html but don't like XML. I think the real reason people don't like XML is it reminds them of Steve Ballmer.

Then there are namespaces, sure there are probably lots of places where you need to use them. But I never encountered a place where they are really needed, but because they are the default you need to work with them or your queries do not work. Super confusing for beginners and annoying as heck.

In the days when XML was popular I've been more active in several Web forums that helped novice users with particular technology (and that included XML). Not a single confusion about XML namespaces came from someone who read the reference. Quoting the reference would be also a very efficient way to clear the confusion.

Bottom line: it's not a problem worth mentioning. In the grand scheme of things an hour you'd have to spend reading the specification is a drop in a bucket compared to all the time you'd have to work with XML. It's a fixed-size effort that you have make once. Compare this to having to deal with bad "number" serialization that you have to deal in JSON every time in a new program that deals with JSON.

Two reasons:

1) Because it's unnecessary complexity. When you add unnecessary complexity into fundamental technology that everything uses, you've now made everything worse. It's like polluting the lake, and then ignoring the fact that beginners need to learn how to boil the water properly drinking it.

2) Because that prevents the technology from being adopted. Whether you think it's justified or not, beginners will choose the tech that's easier to use, and it will succeed.

The market of technology adoption forces us to make things simple for beginners, and in the end, that's good for all of us.

Even without namespaces, it's trivial to handle { "NamespaceA": {....}, "NamespaceB": {....}, }

There is seldom need to mix it in the same object, and if you need that, you should think long and hard if you are on the right track.

As for multiple ones in the same object it makes sense if you want to reuse a definition used elsewhere e.g. to add an address using a predefined address type. It is like using structures/records in programming languages but with no pointers for composition.

Or spend 0 hours reading the JSON reference to reach the same result.

HTML isn’t trying to serialize abstract data and is doing what XML does best in being a document/GUI format. It doesn’t matter all that much that it can’t represent null characters in a standard way because it isn’t a printable character.

Your understanding of "easier" is oversimplified to the point that it's wrong. It's easier to do the wrong thing in JSON, it's harder to do the right thing in JSON (compared to XML).

JSON is a poorly thought-out format. It's problems become progressively more difficult to deal with the more you expect of your program.

  echo '<zero>&#0;</zero>' | xmllint -
  -:1: parser error : xmlParseCharRef: invalid xmlChar value 0
  <zero>&#0;</zero>
            ^

  printf '<?xml version="1.0" encoding="utf-8"?><zero>\0</zero>' | xmllint -
  -:1: parser error : Premature end of data in tag zero line 1
  <?xml version="1.0" encoding="utf-8"?><zero>
                                            ^

  Character Range

  [2]    Char    ::=    [#x1-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */
  [2a]    RestrictedChar    ::=    [#x1-#x8] | [#xB-#xC] | [#xE-#x1F] | [#x7F-#x84] | [#x86-#x9F]

nope

   jq "." <<< "\0"
   parse error: Invalid numeric literal at line 2, column 0

   jq "." <<< '{"name": "\0"}'
   parse error: Invalid escape at line 1, column 13

   jq "." <<< "null"          
   null

   jq "." <<< '{"name": null}'
   {
     "name": null
   }

However, "\u0000" is.

   >> JSON.parse("\u0000")
   Uncaught SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
       <anonymous> debugger eval code:1

\u0000 works fine with firefox with the proper syntax

  JSON.parse(`"\\u0000"`)
  "\u0000"

  printf '{"null":"\u0000"}' | jq
  {
    "null": "\u0000"
  }

of course! I forgot to quote those quotes! (facepalm)

that works. and uses a single byte too.

TIL.

XML on the other hand... Who here can say they actually know anything substantial about XML besides the syntax? My guess is <10%.

JSON is just pure data.

    <foo>something</foo>

    <foo value="something"/>