News for hackers. Stuff that matters.

A comment on DuckDB quacks Arrow: A zero-copy data integration between Arrow and DuckDB. Parent comment

Re:DuckDB quacks Arrow: A zero-copy data integration between... by TheGuyWhoCodes on Friday December 03, 2021 @07:04PM (#29437221)
You could already run SQL on Parquet with DuckDB (even on Java). I believe it already used Arrow under the hood to read the Parquet files, could be wrong tho, but this is a more memory performant integration which is great.
[ Parent | on HN ]
Re:DuckDB quacks Arrow: A zero-copy data integration between... by hfmuehleisen on Saturday December 04, 2021 @12:41AM (#29438987)
DuckDB has its own Parquet reader
[ Parent | on HN ]
Re:DuckDB quacks Arrow: A zero-copy data integration between... by infinite8s on Monday December 06, 2021 @12:43PM (#29462965)
DuckDB can read parquet directly - however the interesting bit is that the results of SQL queries can be returned as arrow objects into python for further processing (by pyarrow/pandas).
[ Parent | on HN ]