News for hackers. Stuff that matters.

A comment on 1.1B Taxi Rides Using DuckDB.

Re:1.1B Taxi Rides Using DuckDB by sleepytimetea on Saturday March 16, 2024 @01:48AM (#39723672)
It would be interesting to see the performance of protobuf vs Parquet in this case...not sure if DuckDB supports that...
[ Parent | on HN ]
Re:1.1B Taxi Rides Using DuckDB by LunaSea on Saturday March 16, 2024 @03:38AM (#39724010)
Not yet, but I believe that Click house does.

Avro would be another contender in the space.

[ Parent | on HN ]
Re:1.1B Taxi Rides Using DuckDB by denny_c on Saturday March 16, 2024 @02:54PM (#39728437)
Parquet is a columnar format with columnar compression. With metadata and indexes. In many cases it needs to read a tiny fraction of Parquet file. Protobuf has zero chances.
[ Parent | on HN ]