PostgreSQL Begins Adding LLVM JIT Support For Faster Performance

Re:PostgreSQL Begins Adding LLVM JIT Support For Faster Perf... by jandrewrogers on Saturday April 07, 2018 @04:42PM (#16782838)

Actually, the performance benefits can be huge (design dependent) and have been for a long time. The reasons have more to do with complexity and difficulty of implementation. The query compilation and execution engine must preserve the operational robustness, integrity, and behavior of the database engine, which is not a trivial design and implementation problem. The number of things that can go horribly wrong increase significantly versus an interpreter.

JIT query compilers were relatively common in closed source databases even a decade ago, it was one of their technical advantages versus open source. This is a necessary step toward closing that gap. There is a similar gap when it comes to storage/execution schedulers that is currently lacking in open source implementations which has a similar level of impact.

[ Parent | on HN ]

Re:PostgreSQL Begins Adding LLVM JIT Support For Faster Perf... by barrkel on Saturday April 07, 2018 @08:46PM (#16783911)

As an example of something that can go wrong: Apache Impala uses LLVM to JIT queries. Turns out some code optimization algorithms are O(n^3) (or worse) or so, where n is small enough not to matter too much with human code, but falls apart with a few thousand columns in a single table, not super unusual with analytics in an environment with expensive joins like Hadoop.

[ Parent | on HN ]