News for hackers. Stuff that matters.
Re:Sq.io: jq for databases and more by pkkm on Monday October 07, 2024 @03:51AM (#41763705)
Sounds like PRQL [1].

[1] https://prql-lang.org/

Re:Sq.io: jq for databases and more by VMG on Monday October 07, 2024 @06:22AM (#41764541)
Kind of. However on first glance it seems like the separator in PRQL is a newline instead of a `|`, which is less ergonomic in a shell.
Re:Sq.io: jq for databases and more by pie_flavor on Monday October 07, 2024 @09:56AM (#41766068)
Newline and pipe mean the same thing and are interchangeable.
Re:Sq.io: jq for databases and more by VMG on Monday October 07, 2024 @11:50AM (#41767214)
nice - are there any PRQL CLI tools for json?
Re:Sq.io: jq for databases and more by snthpy on Monday October 07, 2024 @05:31PM (#41771280)
Not directly, but DuckDB, ClickHouse, and GlareDB all support PRQL and they each have CLIs.

In a previous HN comment [1] I showed how you can leverage PRQL to make your RDBM's JSON functionality more ergonomic. For example

    ```sh
    > prqlc compile <<EOF - | duckdb
    let get = path obj -> s"""{obj} -> {path}"""
    let getstr = path obj -> s"""{obj} ->> {path}"""
    let extract = obj path -> s"""json_extract({obj}, {path})"""
    
    from [{data='{"duck": [1, 2, 3]}'}]
    select { (data | get '$.duck[0]'), (data | getstr '$.duck[1]'), extract data '$.duck[2]'}
    EOF
    ┌───────────────────────┬──────────────────────────┬───────────────────────────────────┐
    │ "data" -> '$.duck[0]' │ ("data" ->> '$.duck[1]') │ json_extract("data", '$.duck[2]') │
    │         json          │         varchar          │               json                │
    ├───────────────────────┼──────────────────────────┼───────────────────────────────────┤
    │ 1                     │ 2                        │ 3                                 │
    └───────────────────────┴──────────────────────────┴───────────────────────────────────┘
    ```
More details in that post.

Unfortunately I don't think this really addresses the grandparent comment though because you're still using jsonpath type expressions to unpack the JSON objects. If you really wanted to use PRQL for everything you would have to first convert and flatten your JSON data into relational tables.

1: https://news.ycombinator.com/item?id=37569946

Re:Sq.io: jq for databases and more by snthpy on Monday October 07, 2024 @05:38PM (#41771337)
Some time ago I wrote pq (https://github.com/prql/prql-query) which aimed to be a simple CLI for PRQL to wrangle data on the command line, much like sq.

Unfortunately I haven't had time to maintain it so it is now archived and out of date. I hope that I might get a chance to update it again. More has happened since then and there are low hanging fruit to make it more usable, for example adding connector_arrow (https://github.com/aljazerzen/connector_arrow) support for other databases, etc...

Quick example of how things looked with pq:

    ```sh
    $ pq --from i=invoices.csv "from i | take 5"
    +------------+-------------+-------------------------------+-------------------------+--------------+---------------+-----------------+---------------------+-------+
    | invoice_id | customer_id | invoice_date                  | billing_address         | billing_city | billing_state | billing_country | billing_postal_code | total |
    +------------+-------------+-------------------------------+-------------------------+--------------+---------------+-----------------+---------------------+-------+
    | 1          | 2           | 2009-01-01T00:00:00.000000000 | Theodor-Heuss-Straße 34 | Stuttgart    |               | Germany         | 70174               | 1.98  |
    | 2          | 4           | 2009-01-02T00:00:00.000000000 | Ullevålsveien 14        | Oslo         |               | Norway          | 0171                | 3.96  |
    | 3          | 8           | 2009-01-03T00:00:00.000000000 | Grétrystraat 63         | Brussels     |               | Belgium         | 1000                | 5.94  |
    | 4          | 14          | 2009-01-06T00:00:00.000000000 | 8210 111 ST NW          | Edmonton     | AB            | Canada          | T6G 2C7             | 8.91  |
    | 5          | 23          | 2009-01-11T00:00:00.000000000 | 69 Salem Street         | Boston       | MA            | USA             | 2113                | 13.86 |
    +------------+-------------+-------------------------------+-------------------------+--------------+---------------+-----------------+---------------------+-------+
    $ # When there is only one input table then this automatically becomes the source relation, i.e. `from i | ` is prepended to the query
    $ # so this can be simplified to:
    $ pq --from invoices.csv "take 5"
    ...
    ```