Zq: An easier and faster alternative to jq

Re:Zq: An easier and faster alternative to jq by weinzierl on Tuesday April 26, 2022 @09:53AM (#31167484)

jq is incredibly powerful and I'm using it more and more. Even better, there is a whole ecosystem of tools that are similar or work in conjunction with jq:

* jq (a great JSON-wrangling tool)

* jc (convert various tools’ output into JSON)

* jo (create JSON objects)

* yq (like jq, but for YAML)

* fq (like jq, but for binary)

* htmlq (like jq, but for HTML)

List shamelessly stolen from Julia Evans[1]. For live links see her page.

Just a few days ago I needed to quickly extract all JWT token expiration dates from a network capture. This is what I came up with:

    fq 'grep("Authorization: Bearer.*" ) | print' server.pcap | grep -o 'ey.*$' | sort | uniq | \
    jq -R '[split(".") | select(length > 0) | .[0],.[1] | gsub("-";"+") | gsub("_";"/") | @base64d | fromjson]' | \
   jq '.[1]' | jq '.exp' | xargs -n1 -I! date '+%Y-%m-%d %H:%M:%S' -d @!

It's not a beauty but I find the fact that you can do it in one line, with proper parsing and no regex trickery, remarkable.

[1] https://jvns.ca/blog/2022/04/12/a-list-of-new-ish--command-l...

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by kitd on Tuesday April 26, 2022 @10:45AM (#31168116)

Also highly recommended is gron [0], to make json easily searchable

[0] https://github.com/TomNomNom/gron

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by spudlyo on Tuesday April 26, 2022 @11:37AM (#31168838)

Most of the time I can get what I need with gron and traditional UNIX tools, without needing to reach for jq, and without having to re-learn its somewhat arcane syntax.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by zikduruqe on Tuesday April 26, 2022 @10:55AM (#31168233)

I came here looking for a gron recommendation. I use this very often.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by toxik on Tuesday April 26, 2022 @03:24PM (#31171977)

Is your example not easier to write and read as a 10-something line Python script? I never understood the appeal of jq etc because of this very reason.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by hoherd on Tuesday April 26, 2022 @11:29AM (#31168709)

I would definitely add dasel to that list. It's become my de facto serialized data converter, and regularly use it to convert between csv, toml, yaml, json, and xml using jq-ish syntaxes.

https://github.com/tomwright/dasel

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by chriswarbo on Tuesday April 26, 2022 @11:11AM (#31168455)

The yq tool also provides 'xq', which works on XML :)

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by stormbrew on Tuesday April 26, 2022 @09:54PM (#31175369)

tbh my biggest problem with all these tools is that I really don't want to have to learn one for each of the json-y formats I have to use every day. If jq supported toml and yaml natively I'd be much much much happier to learn its kind of obtuse syntax.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by samcal on Tuesday April 26, 2022 @10:44PM (#31175682)

Have you checked out https://www.nushell.sh/? It seems like exactly what you're describing. Although I know of people who are happily using it as their main shell, I only really use it when I need to read and manipulate data in files.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by wwader on Wednesday April 27, 2022 @04:03AM (#31177510)

Hi, lover of jq and author of fq here! Just wanted to mention that fq is jq so you can do things like `fq 'grep("Authorization: Bearer.*") | split("\n") | ...' file.pcap`.

Also i'm working and prototyping some kind of http decoding support that will make things like select on headers and automatic/manual decoding body possible.

[ Parent | on HN ]

Re:Zq: An easier and faster alternative to jq by chris37879 on Tuesday April 26, 2022 @10:06PM (#31175437)

I need someone to make a `Q` wrapper that amalgamates all of them. And if that's already taken by a common utility, I vote we name it deLancie, instead.

[ Parent | on HN ]