News for hackers. Stuff that matters.

A comment on Show HN: Hydra 1.0 – open-source column-oriented Postgres. Parent comment

Re:Show HN: Hydra 1.0 – open-source column-oriented Postgres by zX41ZdbW on Thursday August 03, 2023 @05:23PM (#36992152)
ClickHouse can do large GROUP BY queries, not limited by memory: https://clickhouse.com/docs/en/sql-reference/statements/sele...
[ Parent | on HN ]
Re:Show HN: Hydra 1.0 – open-source column-oriented Postgres by riku_iki on Thursday August 03, 2023 @05:25PM (#36992186)
as explained in https://github.com/ClickHouse/ClickHouse/issues/47521#issuec... it can't, that parameters only applies on pre aggregation phase but not aggregation.

Feature request is not implemented yet: https://github.com/ClickHouse/ClickHouse/issues/40588

[ Parent | on HN ]
Re:Show HN: Hydra 1.0 – open-source column-oriented Postgres by zX41ZdbW on Friday August 04, 2023 @07:47AM (#36998595)
ClickHouse uses "grace hash" GROUP BY with the number of buckets = 256.

It can do size about 256 times larger than a memory because only one bucket has to be in memory while merging. It works for distributed query processing as well and is enabled by default.

About the linked issue - it looks like it is related to some extra optimization on top of what already exists.

[ Parent | on HN ]
Re:Show HN: Hydra 1.0 – open-source column-oriented Postgres by riku_iki on Friday August 04, 2023 @01:02PM (#37002701)
> only one bucket has to be in memory while merging.

its hard for me to judge about implementation details, but per that person reply memory is also multiplied by number of threads which do aggregation.

[ Parent | on HN ]