Parsing JSON is a Minefield

Re:Parsing JSON is a Minefield by jstimpfle on Sunday April 22, 2018 @02:19PM (#16897609)

Xml sort-of died (in many domains) because of its insane complexity and its redundant ways of specifying relations (child relationships vs explicit relationships, tag name data, attributes data, body data), lack of legibility, parser performance (probably an inherent problem due to hierarchical representation?) and other issues like even meaning of whitespace. >50% of the JSON or Xml I've seen would actually be much easier to read and have much clearer semantics, if just written as a relational database. Some time ago, I tried to improve on CSV by specifying it better and making it more powerful. The result was not too bad: http://jstimpfle.de/projects/wsl/main.html . But I think it should be trimmed down even more, and have only built-in datatypes like JSON, to be able to replace it. (More ambitious standardization efforts would lead to similar problems as with Xml, I think). That's why so far I use the approach only in ad-hoc implementations, in different flavours needed for various tasks.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by ProblemFactory on Sunday April 22, 2018 @02:55PM (#16897830)

XML parsers are necessarily over-complicated for structured data, because it is a text markup language, not a nested data structure language.

<address>123 Hello World Road, <postcode>12345</postcode>, CA</address>

is perfectly sensible XML. The address is not a tree structure or a key-value dictionary - it is free text with optional markup for some words.

You can use XML to represent nested data structures with lists and dictionaries, but the parsers and their public APIs must still handle the freeform text case as well.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by jstimpfle on Sunday April 22, 2018 @03:00PM (#16897855)

Yep, the application to text documents is valid in my eyes, as well. Although there are lighter weight and/or more extensible approaches, like TeX. (update, clarificaton: I mean just the markup syntax, not the compuational model)

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by anonymouz on Sunday April 22, 2018 @03:33PM (#16898058)

> TeX

Dear god no. I use and love (La)TeX daily to write documents. But as a markup format for data that's supposed to be processed in any way, other than being fed to a TeX engine, it's absolutely terrible. You can't even really parse a TeX document; with all the macros it really is more a program than a document. XML is far from perfect, but it works well as a markup and data exchange format that is well-specified.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by _delirium on Sunday April 22, 2018 @03:30PM (#16898040)

I like TeX for producing documents. But I'd take XML over TeX if I had to parse the markup myself, outside of the TeX toolchain. Any nontrivial TeX document is built out of a pile of macros, so you need to implement a TeX-compatible macro expander to parse it. And at least with XML there are solid libraries, while the state of TeX-parsing libraries outside of TeX itself is pretty poor. I think Haskell is the only language with a reasonably good implementation, thanks to the efforts of the pandoc folks.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by jstimpfle on Sunday April 22, 2018 @03:59PM (#16898216)

It doesn't need to be Tex-compatible. My point was just that the syntax is lighter weight and might be preferable for some applications.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by ygra on Sunday April 22, 2018 @09:30PM (#16899853)

RTF is essentially the same syntax, sans the option to define your own markup. Only barely human readable when produced by a word processor, though, but generated TeX is awful as well.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by Someone on Sunday April 22, 2018 @04:34PM (#16898431)

Lighter-weight? Tex is Turing-complete. You can’t even know whether interpreting it will ever finish, and writing a parser that produces good error messages on invalid input is difficult.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by edejong on Sunday April 22, 2018 @04:49PM (#16898500)

From someone who has written TeX macro’s before: you probably mistake the ‘clean’ environment of LaTeX with the core TeX language. The former is reasonable, if very limited, the latter is die-hard “you thought you knew how to program, but this proves you wrong”-material.

XML over TeX any time and LISP-like over XML (with structural macros)

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by kazinator on Sunday April 22, 2018 @08:26PM (#16899574)

I would make a subtle adjustment there: "you thought Knuth knew how to design, this proves you wrong". \makeatspecial here we go\oh\boy ...

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by edejong on Monday April 23, 2018 @04:22PM (#16907038)

Ah, yes, agreed! That was my first thought as well when I went into the deep flaming pit of TeX macros. Nevertheless, I guess he didn't have much reference material back then or did he?

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by jstimpfle on Monday April 23, 2018 @05:15PM (#16907481)

FWIW, I remember a quote about Knuth and TeX, "He tried really not to make it a programming language, but ultimately he failed". I don't remember who said that.

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by svat on Monday April 23, 2018 @09:32PM (#16909127)

You're probably thinking of Knuth himself. He's mentioned several times how he never intended to make a programming language, and how puzzled he is that people write programs in TeX macros.

E.g.:

> In some sense I put in many of the programming features kicking and screaming [...] Every system I looked at had its own universal Turing machine built into it somehow, and everybody’s was a little different from everybody else’s. So I thought, “Well, I’m not going to design a programming language; I wanted to have just a typesetting language.” Little by little, I needed more features and so the programming constructs grew.[...] as a programmer, I was tired of having to learn ten different almost-the-same programming languages for every system I looked at; I was going to try to avoid that.

etc. (https://www.ntg.nl/maps/16/15.pdf)

and:

> I was really thinking of TeX as something that the more programming it had in it, the less it was doing its real mission of typesetting. When I put in the calculation of prime numbers into the TeX manual I was not thinking of this as the way to use TeX. I was thinking, "Oh, by the way, look at this: dogs can stand on their hind legs and TeX can calculate prime numbers."

(Coders at Work interview)

In fact, if you use TeX the way Knuth intended and uses it, then the use of macros or programming is really quite minimal. It's only LaTeX that to pursue a better document interface for the user, ends up writing horrifically complex macros -- Mittelbach mentions that nine out of ten "dirty tricks" mentioned by Knuth in the TeXbook are actually used in the source code of LaTeX!

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by dagenix on Monday April 23, 2018 @01:55AM (#16900835)

"Insane complexity"? You mean that it actually has a spec instead of the back of a business card with no implementations that agree on what is actually valid?

[ Parent | on HN ]

Re:Parsing JSON is a Minefield by jstimpfle on Monday April 23, 2018 @03:59AM (#16901382)

Yes. No. What it does is too complicated. And I hear that consistent implementations are not a reality for Xml, either (at least regarding implemented features).

[ Parent | on HN ]