News for hackers. Stuff that matters.
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by theLiminator on Tuesday April 22, 2025 @04:35PM (#43766011)
How does it compare to duckdb and/or polars?
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by thenaturalist on Tuesday April 22, 2025 @06:41PM (#43766982)
This is very much an active space, so the half-life of in depth analyses is limited, but one of the best write ups from about 1.5 years ago is this one: https://bicortex.com/duckdb-vs-clickhouse-performance-compar...
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by nasretdinov on Wednesday April 23, 2025 @02:50AM (#43769335)
In my understanding DuckDB doesn't have its own optimised storage that can accept writes (in a sense that ClickHouse does, where it's native storage format gives you best performance), and instead relies on e.g. reading data from Parquet and other formats. That makes sense for an embedded analytics engine on top of existing files, but might be a problem if you wanted to use DuckDB e.g. for real-time analytics where the inserted data needs to be available for querying in a few seconds after it's been inserted. ClickHouse was designed for the latter use case, but at a cost of being a full-fledged standalone service by design. There are embedded versions of ClickHouse, but they are much bulkier and generally less ergonomic to use (although that's a personal preference)
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by theLiminator on Wednesday April 23, 2025 @01:48PM (#43774727)
Nah, duckdb does have their own format (not sure if it's write friendly, though i believe it is).
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by nasretdinov on Wednesday April 23, 2025 @01:51PM (#43774760)
It does, but the performance isn't great apparently: https://github.com/duckdb/duckdb/discussions/10161
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by theLiminator on Wednesday April 23, 2025 @05:21PM (#43776740)
Yeah, I don't really know. Though in the OLAP space that issue/discussion is really old. There's a good chance that performance is dramatically better now though YMMV.
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by thenaturalist on Thursday April 24, 2025 @11:39AM (#43784065)
It cannot do concurrent writes natively and that is not a design goal of the DuckDB creators. See: https://duckdb.org/docs/stable/connect/concurrency.html#writ...
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by spatular on Thursday April 24, 2025 @11:43AM (#43784115)
Clickhouse is a network server, duckdb and polars are in-process databases. It's like postgres vs sqllite.
Re:ClickHouse gets lazier (and faster): Introducing lazy mat... by theLiminator on Friday April 25, 2025 @12:59PM (#43795989)
There's chdb though.