News for hackers. Stuff that matters.

co0lster

duckdb relies on filter pushdowns that makes the reason harder to spot, but you don’t use sqlite to connect to database via JDBC protocol, are you?
You spawn in memory instance of duckdb and connect to it.
650GB relates to size of parquet files which are compressed in reality it’s way more.

32 GB of parquet cannot fit in 32GB of RAM

I’m stealing Spotify random instrumental music idea! Why I cannot come up with similar ideas by my own? Thats the hardest part I think…