News for hackers. Stuff that matters.
Re:jq 1.7 released by hn92726819 on Wednesday September 06, 2023 @11:16PM (#37414337)
> I hate that the jq command is always some indecipherable string in the middle of the script

It might be worthwhile to just learn how jq works. At the end of the day, you need to learn some language to parse json. I hate DSLs too, but I cannot think of anything as useful and concise as jq.

> but that ends up being at least as nasty as the JQ script

That's exaxtly why jq is so nice. Nice alternatives just don't exist

Re:jq 1.7 released by speed_spread on Wednesday September 06, 2023 @11:43PM (#37414507)
Something not having alternatives doesn't make it necessarily nice. It's okay to wish for better even if you have something that works.
Re:jq 1.7 released by ustolemyname on Thursday September 07, 2023 @02:06AM (#37415294)
I think in this case "better" reduces to convincing the upstream data source to not use json.

Putting that frustration on jq seems like a case of transference.

Re:jq 1.7 released by spion on Thursday September 07, 2023 @07:56AM (#37417636)
Or maybe jq does have some design flaws.
Re:jq 1.7 released by Nullabillity on Thursday September 07, 2023 @08:35AM (#37418005)
And now you've turned a JSON traversal problem into a parsing problem.. congratulations?
Re:jq 1.7 released by hn92726819 on Friday September 08, 2023 @01:38PM (#37436773)
> Something not having alternatives doesn't make it necessarily nice

Of course not, but compared to every alternative today, jq is eons better than everything else. It's conciseness, ease of use, ease of learning all make it awesome. So as of right now, it is the nicest thing to use by far.

Personally though, I don't think I do wish for better. Jq is missing nothing that I want.

Re:jq 1.7 released by p-e-w on Thursday September 07, 2023 @02:22AM (#37415383)
> That's exaxtly why jq is so nice. Nice alternatives just don't exist

Write a simple Python script, parse JSON into native objects, manipulate those objects as desired with standard Python code, then serialize back into JSON if necessary. Voila, you have a readable, maintainable, straightforward solution, and the only dependency (the Python interpreter) is already preinstalled on almost every modern system.

Sure, you may need a few more lines of code than what would be possible with a tailor-made DSL like jq, but this isn't code golf. Good code targets humans, not "least possible number of bytes, arranged in the cleverest possible way".

Re:jq 1.7 released by hnben on Thursday September 07, 2023 @03:08AM (#37415652)
> Sure, you may need a few more lines of code than ...

jQ integrates very nicely into bash script. Especially in between pipes a short&simple jq-snippet can work wonders for readability of the overall script.

On the other hand, if the bash script becomes too complex it may be a good idea to replace the entire bash script with python (instead of just the json-parsing-part)

Re:jq 1.7 released by viraptor on Thursday September 07, 2023 @03:16AM (#37415701)
> Especially in between pipes a short&simple jq-snippet

Many of them are not short and simple though. And each time you do a some transformation, you pretty much need to go in/out of jq at each step of it want to make some decisions or get multiple types of results without processing the original multiple times.

Re:jq 1.7 released by p-e-w on Thursday September 07, 2023 @03:47AM (#37415903)
> a short&simple jq-snippet can work wonders for readability of the overall script.

... if the reader happens to be familiar with the niche language "jq".

Otherwise, you may as well have put some Akkadian cuneiform in there.

Re:jq 1.7 released by Ekaros on Thursday September 07, 2023 @06:50AM (#37417112)
So awk or sed or maybe even grep?

jq seems slightly better than those...

Re:jq 1.7 released by cryptonector on Thursday September 07, 2023 @10:19AM (#37419254)
Certainly for JSON.
Re:jq 1.7 released by mmcdermott on Thursday September 07, 2023 @05:43PM (#37426014)
> ... if the reader happens to be familiar with the niche language "jq".

Eh. Linux/Unix has always had an affinity for DSLs and mini-languages. If you're willing to work with bash, sed, awk, perl, lex, yacc, bc/dc etc. jq doesn't seem like it should cause too much consternation.

Re:jq 1.7 released by ianleeclark on Thursday September 07, 2023 @03:30AM (#37415795)
The point in my career at which I used jq the most was when I was doing a lot of work with Elasticsearch doing exploratory work on indexed data and search results. Doing things such as trying to figure out what sort of values `key` might have, grabbing ids returned, etc.

Second to this, I've mostly used jq to look at OpenAPI/swagger files, again just doing one-off tasks, such as listing all api routes, listing similarly named schemas, etc.

From what I've seen in the companies I've worked for, this is fairly consistent, but naturally I can't speak for everyone's use-cases. At the end of the day, I don't think most people use jq in places where readable or maintainable would be most appropriate.

Re:jq 1.7 released by darkwater on Thursday September 07, 2023 @03:33AM (#37415809)
The simple existence of DSL tools like jq is the testament to the fact that people don't want to go to a generic language to solve every kind of problem. I'm also convinced that a big subset of "use generic language for everything" do it because they want to use their shiny hammer on that nail as well.
Re:jq 1.7 released by 6510 on Friday September 08, 2023 @02:32PM (#37437527)
If I don't use something often enough I just forget how it works.
Re:jq 1.7 released by rwmj on Thursday September 07, 2023 @03:51AM (#37415931)
Re:jq 1.7 released by comprev on Thursday September 07, 2023 @04:29AM (#37416197)
jq can be easily downloaded, unpacked, executed and deleted in one line of bash.

It's beauty is the simplicity and portability.

Re:jq 1.7 released by datatrashfire on Thursday September 07, 2023 @10:27AM (#37419376)
Yea except the python solution is probably going to be several hundred lines, instead of a few.

Python is often not installed in server environments unless it's a runtime environment for Python.

Want to use a non standard library? Now your coworkers are suddenly in Python dependency hell. Better hope anyone else that wants to use this is either familiar with the ecosystem, or just happens to have an identical runtime environment as you.

Or someone could just curl/apt/dnf a jq binary to use your 3 line query, instead of maintaining all of this + 200 lines of Python.

Re:jq 1.7 released by wwader on Thursday September 07, 2023 @10:33AM (#37419468)
I do lots of exploratory work in various structure data, in my case often debugging media filea via https://github.com/wader/fq, which mean doing lots of use-once-queries on the command line or REPL. In those cases jq line-friendly and composable syntax and generators really shine.
Re:jq 1.7 released by hn92726819 on Friday September 08, 2023 @01:34PM (#37436705)
I got to jq for the same reason I go to regular expressions. If you tell me this is too complex

    (?:[A-Z][a-z]+_?(\d+))
Then I don't know what to tell you. Do you think that's too complex and should be a python script too? I don't think so. It looks complex, but if you just learn it, it's easier than a 'simple' script to do the same thing.

I'd argue it's good code if you don't have to sift through lines of boilerplate to do something so trivial in jq or regex syntax.

Re:jq 1.7 released by ahoka on Thursday September 07, 2023 @06:36AM (#37417016)
What about JSONPath?
Re:jq 1.7 released by mdaniel on Thursday September 07, 2023 @12:15PM (#37421222)
It may be fine for hello-world stuff, but how would one JSONPath this?

  echo '[{"name":"_skip"},{"name":"alpha"},{"name":"_other"}' | \
    jq '[ .[] | select(.name|test("^_.*")|not) | . ]'
The same is roughly true for JMESPath, also, although at least it does actually try to allow projections and some limited functions
Re:jq 1.7 released by storoj on Friday September 08, 2023 @05:44AM (#37431561)
Is ".*" necessary? I think "^_" should be enough.
Re:jq 1.7 released by ahoka on Saturday September 09, 2023 @01:15PM (#37447682)
I believe like this.

  $[?(@.name =~ /[^_].*/)]
Re:jq 1.7 released by mdaniel on Saturday September 09, 2023 @02:02PM (#37448260)
Interesting, I only knew about <https://goessner.net/articles/JsonPath/index.html> which makes no reference to =~ nor any such // syntax but it seems there's an IETF draft that is (a) an actual specification (b) includes more modern stuff like what you used: https://www.ietf.org/archive/id/draft-ietf-jsonpath-base-07....

And, while whatever is powering https://jsonpath.com/ does honor your syntax, albeit with an absolutely useless result:

    [
      {
        "name": -1
      },
      {
        "name": -1
      },
      {
        "name": -1
      }
    ]
I found that `pip install jsonpath-ng` does not accept it nor mention it <https://github.com/h2non/jsonpath-ng/tree/v1.5.3?tab=readme-...> so I think it's out on the bleeding edge or something
Re:jq 1.7 released by hn92726819 on Friday September 08, 2023 @01:42PM (#37436841)
I hadn't seen this before. At a quick glance, the syntax looks fine. Though I don't know what command line utility I'd need to use it. It makes me wonder how hard a translator from jq syntax to jsonpath would be... Then we could have our cake and eat it too.
Re:jq 1.7 released by jordibc on Thursday September 07, 2023 @11:08AM (#37420034)
I really like jq, but I think there is at least one nice alternative to it: jet [1].

It is also a single executable, written in clojure and fast. Among other niceties, you don't have to learn any DSL in this case -- at least not if you already know clojure!

[1] https://github.com/borkdude/jet