Re:The missing tier for query compilers
by zX41ZdbW on Wednesday February 12, 2025 @01:17PM
(#43028078)
This is an interesting article!
ClickHouse does a hybrid approach. It uses a vectorized interpreter plus the JIT compilation (with LLVM) of small expressions (which is also vectorized and generated for the target instruction set by compiling loops). The JIT compilation does not bother to compile the whole query, only simple fragments. It gives only a marginal improvement over the vectorized interpreter.
ClickHouse does a hybrid approach. It uses a vectorized interpreter plus the JIT compilation (with LLVM) of small expressions (which is also vectorized and generated for the target instruction set by compiling loops). The JIT compilation does not bother to compile the whole query, only simple fragments. It gives only a marginal improvement over the vectorized interpreter.
Even with this approach, JIT is a huge pile of problems. A few details here: https://clickhouse.com/blog/clickhouse-just-in-time-compiler...
Overall architecture overview: https://clickhouse.com/blog/first-clickhouse-research-paper-...