News for hackers. Stuff that matters.

A comment on Fast analysis with DuckDB and Pyarrow. Parent comment

Re:Fast analysis with DuckDB and Pyarrow by vanpythonista on Sunday May 01, 2022 @01:59AM (#31222589)
Shameless plug, my project exposes the common Parquet operations using a Rust CLI tool that utilizes the Rust API for Apache Arrow and can be used without any Java/Hadoop/Spark dependencies. Also available a static binary.

https://github.com/manojkarthick/pqrs

[ Parent | on HN ]
Re:Fast analysis with DuckDB and Pyarrow by 41b696ef1113 on Sunday May 01, 2022 @03:12AM (#31222867)
That looks great! Exactly the kind of tooling required to allow me to feel more comfortable locking up human readable data into a blob. Being a single Rust binary is just the cherry on top.

I might submit a few feature requests, but one that immediately comes to mind: csv -> parquet. Perhaps out of scope for the original vision, but having a single utility that could roundtrip data would be fantastically useful.

[ Parent | on HN ]
Re:Fast analysis with DuckDB and Pyarrow by vanpythonista on Sunday May 01, 2022 @07:34AM (#31224153)
Glad you like it. A CSV->Parquet conversion command would certainly be really useful!
[ Parent | on HN ]