News for hackers. Stuff that matters.

A comment on Tips on adding JSON output to your CLI app.

Re:Tips on adding JSON output to your CLI app by IgorPartola on Sunday December 05, 2021 @08:14PM (#29455080) 

    "kb_read_s": 0.12
This worries me. JSON doesn’t have support for fixed point math, does it? When will some random POSIX tool spit out scientific notation at me.

Also, if you just output a flat schema, is there much of a point in this vs just:

    cpu: 0.2
    kw_read_s: 0.12
The difference is that you can use a JSON parser vs splitting on new lines and colons?

I do like the idea of JSON output as an option but before every bug and mistake gets canonized as POSIX or some other standard can we at least talk about the output format for a bit?

[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by zeroimpl on Sunday December 05, 2021 @09:18PM (#29455482)
You'll prefer JSON the minute it becomes:

    cpu: 0.2
    kb_read_s: 0.12
    mac_addr: 10:AA:FF:00:55:66
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Monday December 06, 2021 @12:05AM (#29456364) 

    dict((key.strip(), val.strip()) for key, _, val in line.strip().partition(‘:’) for line in text.split(‘\n’) if line and line.strip())
Also who says we can’t have a universal parser for this format just like we have for JSON? Not everyone needs to write the one liner like above, just use the libtextformat.parse(text) or whatever we would call it.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by wruza on Monday December 06, 2021 @01:20AM (#29456775)
Parsing outputs is a point where unix philosophy always breaks down for me. It looks like this:

  {program -> human-text -> parser}+
When it could look like this:

  program
    -> {struct-text -> program}+
    [-> formatter] -> human-text .
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Monday December 06, 2021 @09:40AM (#29460229)
I don’t think your approach is a problem. I do think that JSON might be not the ideal format for struct text. Especially if you really just use it as a flat dict of key value pairs.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by kbrazil on Monday December 06, 2021 @11:28AM (#29461747)
Maybe not, but more often than not you really have an array of objects, and that's where JSON helps.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Monday December 06, 2021 @07:20PM (#29467621)
Look at my original example. The JSON punctuation just muddled the text format without adding any additional value. The point of JSON is the (very limited) type system, nested schema, and heterogeneous format (if array A has two elements B and C, they don’t need to be the same type). None of that seems applicable to what you want out of a UNIX utility.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by kbrazil on Tuesday December 07, 2021 @12:36PM (#29475588)
What I was getting at is:

    cpu: 0.2
    kw_read_s: 0.12
is describing a single 'object'. How do you describe an array or list of these objects? Something like:

    cpu: 0.2
    kw_read_s: 0.12

    cpu: 1.3
    kw_read_s: 0.4
Do you use a blank line to denote a new 'object'? In JSON it would be done this way, and this conforms to how the vast majority of command output maps (as they tend to be rows of columnar data):

    [
      {
        "cpu": 0.2,
        "kw_read_s": 0.12
      },
      {
        "cpu": 1.3,
        "kw_read_s": 0.4
      }
    ]
The other benefit to JSON here is that the formatting doesn't matter. This could also be expressed as a block of text with no spaces or newlines between elements:

    [{"cpu": 0.2,"kw_read_s": 0.12},{"cpu": 1.3,"kw_read_s": 0.4}]
Finally, this can also be streamed, using JSON Lines:

    {"cpu": 0.2, "kw_read_s": 0.12}
    {"cpu": 1.3, "kw_read_s": 0.4}
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Tuesday December 07, 2021 @08:09PM (#29480241)
The OP specifically advocates against your proposed pattern and points out why nesting isn’t ideal. Also JSON doesn’t support fixed point numbers so JSON.encode(JSON.parse(input)) != input even if your equality assumes that formatting doesn’t matter.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by zeroimpl on Monday December 06, 2021 @09:50PM (#29468642)
You're not wrong, but sometimes simple formats aren't as simple as they look. It sounds like what you want is basically a format like /etc/os-release:

    NAME="Rocky Linux"
    VERSION="8.5 (Green Obsidian)"
    ID="rocky"
    ID_LIKE="rhel centos fedora"
    VERSION_ID="8.5"
    PLATFORM_ID="platform:el8"
    PRETTY_NAME="Rocky Linux 8.5 (Green Obsidian)"
    ANSI_COLOR="0;32"
    CPE_NAME="cpe:/o:rocky:rocky:8.5:GA"
    HOME_URL="https://rockylinux.org/"
    BUG_REPORT_URL="https://bugs.rockylinux.org/"
    ROCKY_SUPPORT_PRODUCT="Rocky Linux"
    ROCKY_SUPPORT_PRODUCT_VERSION="8"
On the surface this seems great, but those quotation marks are kind of annoying. Is it possible there's an escape syntax that's used in case the name also includes quotes? eg VERSION="8.5 (Green \"Aqua\" Obsidian)"? Is it also possible you can embed newlines in between the quotes too? Who knows... Thankfully with JSON there is a simple spec.
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by madeofpalk on Monday December 06, 2021 @05:56AM (#29458210) 

    json.parse(input)
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Monday December 06, 2021 @09:37AM (#29460201)
Memory inefficient.

Type support is lacking. Not date support for example.

Unnecessary quotes around keys make it harder for humans to read.

Different implementations allow for repeated keys.

    otherformat.parse(input) // just as reasonable
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by EuAndreh on Sunday December 05, 2021 @10:57PM (#29455961) 

  cut -d: -f2-
[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by toomanybeersies on Monday December 06, 2021 @03:07AM (#29457358)
> JSON doesn’t have support for fixed point math

Plain text doesn't have support for numbers at all, which isn't much of a solution.

[ Parent | on HN ]
Re:Tips on adding JSON output to your CLI app by IgorPartola on Monday December 06, 2021 @07:36PM (#29467731)
Right. So if we replace plain text then let’s maybe do better than JSON or not do it at all.

Let’s put it this way: if I proposed XML as the substitution for plain text, would you rather keep plain text or switch to XML?

[ Parent | on HN ]