News for hackers. Stuff that matters.

A comment on 1.1B Taxi Rides Using DuckDB.

Re:1.1B Taxi Rides Using DuckDB by tosh on Saturday March 16, 2024 @04:36AM (#39724255)
How is query performance on DuckDB’s native format compared to parquet?
[ Parent | on HN ]
Re:1.1B Taxi Rides Using DuckDB by szarnyasg on Saturday March 16, 2024 @08:02AM (#39725136)
I wrote a page about this: https://duckdb.org/docs/guides/performance/file_formats

TL;DR: if your workload is aggregation-heavy, Parquet is pretty good. If you have many joins, the lack of fine-grained statistics will likely result in bad join plans.

(Disclaimer: I work at DuckDB Labs)

[ Parent | on HN ]