News for hackers. Stuff that matters.
Re:Querying Parquet with Precision Using DuckDB by wenc on Friday June 25, 2021 @05:46PM (#27636670)
Drill is a fair choice (it supports tons of formats), but for me it was a bit heavy -- it required Java and plus it returned results in a form that wasn't easily manipulated (the result is a table in the console drawn with text-chars [1]). If you had a really long result set, you couldn't do a buffered scroll due to text UI limitations -- everything gets printed out and you have to either do manual pagination using LIMIT clauses or just have a really long scrollback buffer. This kind of breaks the continuity of many data analysis workflows.

With DuckDB, the return result is a Python data structure (list of tuples), which is amenable to further manipulation without exporting to another format.

Many GUI db tools often return results in a tabular form which can be copy-pasted into other GUI apps (or copy-pasted as CSV).

[1] https://drill.apache.org/docs/drill-in-10-minutes/#querying-...

Re:Querying Parquet with Precision Using DuckDB by sixdimensional on Saturday June 26, 2021 @12:15AM (#27639055)
You didn't connect to drill using JDBC? You can use any open source GUI with that, like DBeaver.
Re:Querying Parquet with Precision Using DuckDB by wenc on Saturday June 26, 2021 @01:11AM (#27639293)
Oh I didn't think to do that. That's something I could definitely explore, particularly for apps like Tableau. Thanks!