apavlo

ClickHouse | Remote | Full-time | https://clickhouse.com/

ClickHouse Labs is an applied research group that works closely with engineering teams to identify challenging problems in database systems and develop novel solutions. Our goal is to advance the state of the art and translate research ideas into real-world system improvements.

I am looking for database / systems / AI researchers to help establish a world-class research team:

* Research Scientist (all levels): https://jobs.ashbyhq.com/clickhouse/962f45f3-d8d3-4073-add2-...

----

More info: https://clickhouse.com/blog/andy-pavlo-joins-clickhouse

HN discussion: https://news.ycombinator.com/item?id=49156011

There are several out of date entries in there. We used to have students help with writing entries. I just did a major refresh of the site this summer. I need to go through all the entries and figure out how to update them.

Remy - Good to hear from you. Yes things are mess right now. Let me get settled with setting up the lab and we'll figure out how we want to engage with academia.

TBD

They will continue. New seminar series starts next month (announcement coming this week).

I'm not sure what you think "research lab" means? As I mentioned in the article, we look at IBM Research (Almaden) and Microsoft Research as inspiration.

I've never heard of the "Willow" data model before. It looks just like the hierarchical data model (restricted to two levels) from the 1960s with timestamp versioning.

It is not clear what problem this is trying to solve that can't be handled by the relational model.

For those that are unaware, the instructor of this is on the board of OpenAI:

https://openai.com/index/zico-kolter-joins-openais-board-of-...

> I'd love to hear about semantic layer developments in this space (e.g. Malloy etc.)

We also hosted Llyod to give a talk about Malloy in March 2025:

https://db.cs.cmu.edu/events/sql-death-malloy-a-modern-open-...

> How is SpacetimeDB not mentioned here?

https://www.cs.cmu.edu/~pavlo/blog/2026/01/2025-databases-re...

> none of the reviews of the last few years mention immutable and/or bi-temporal databases.

We hosted XTDB to give a tech talk five weeks ago:

https://db.cs.cmu.edu/events/futuredata-reconstructing-histo...

> Which looks more like a blind spot to me honestly.

What do you want me to say about them? Just that they exist?

> Am I living in a bubble?

There are rumblings that the MySQL project is rudderless after Oracle fired the team working on the open-source project in September 2025. Oracle is putting all its energy in its closed-source MySQL Heatwave product. There is a new company that is looking to take over leadership of open-source MySQL but I can't talk about them yet.

The MariaDB Corporation financial problems have also spooked companies and so more of them are looking to switch to Postgres.

> Nothing about time series-oriented databases?

https://www.cs.cmu.edu/~pavlo/blog/2026/01/2025-databases-re...

> I can't believe that article has no mention of SQLite ??

https://www.cs.cmu.edu/~pavlo/blog/2026/01/2025-databases-re...

Thanks for catching this. Updated: https://www.cs.cmu.edu/~pavlo/blog/2026/01/2025-databases-re...

I need to figure out an automatic way to track these.

> If the org behind it ever decides to rugpull/elastic you

I love it that you use "elastic" as a verb here.

> so... you take 10%-30% performance hit _right away_, and you perpetually give up any opportunities to improve the decoder in the future.

The WASM is meant as a backup. If you have the native decoder installed (e.g., as a crate), then a system will prefer to use that. Otherwise, fallback to WASM. A 10-30% performance hit is worth it over not being able to read a file at all.

The backstory is complicated. The plan was to establish a consortium between CMU, Tsinghua, Meta, CWI, VoltronData, Nvidia, and SpiralDB to unify behind a single file format. But that fell through after CMU's lawyers freaked out over Meta's NDA stuff to get access to a preview of Velox Nimble. IANAL, but Meta's NDA seemed reasonable to me. So the plan fell through after about a year, and then everyone released their own format:

→ Meta's Nimble: https://github.com/facebookincubator/nimble

→ CWI's FastLanes: https://github.com/cwida/FastLanes

→ SpiralDB's Vortex: https://vortex.dev

→ CMU + Tsinghua F3: https://github.com/future-file-format/f3

On the research side, we (CMU + Tsinghua) weren't interested in developing new encoders and instead wanted to focus on the WASM embedding part. The original idea came as a suggestion from Hannes@DuckDB to Wes McKinney (a co-author with us). We just used Vortex's implementations since they were in Rust and with some tweaks we could get most of them to compile to WASM. Vortex is orthogonal to the F3 project and has the engineering energy necessary to support it. F3 is an academic prototype right now.

I note that the Germans also released their own fileformat this year that also uses WASM. But they WASM-ify the entire file and not individual column groups:

→ Germans: https://github.com/AnyBlox

> • Memory-Mapping (mmap): We treat the database file as if it’s already in memory, eliminating the distinction between disk and RAM.

Ugh, not another one...

You are conflating MySQL and InnoDB. The latter does a lot of good things, much more than the former.

They double dip. You pay them to review your company. Other companies pay them to read those reviews.

Dude fix your DBMS implementation before you start losing people's data. Or switch to something vetted like SQLite.

This schedule is not finalized yet.

> I'm gonna have to implement my own buffer pool and manage my reads directly like that vexatious paper[1] said all along.

You make it sound like I was trying to troll everyone when we wrote that paper. We were warning you.

"Bridged Indexes" is a non-standard term. These are just secondary indexes using logical pointers with a mapping index. IIRC, Oracle, Hana, and HyPer do the same thing.

Source: https://db.cs.cmu.edu/papers/2017/p781-wu.pdf (see Table 1 + Section 6.1)

> It's quite amazing how a db like this shows that all of those row-based dbs are doing something wrong

They're not "doing something wrong". They are designed differently for different target workloads.

Row-based -> OLTP -> "Fetch the entire records from order table where user_id = XYZ"

Column-based -> OLAP -> "Compute the total amount of orders from the order table grouped by month/year"

> Any opensource libraries in that satellite's tech stack will now get to brag about "our code running on the moon" :)

A safe bet would be that SQLite is on there. It's already in airplanes / satellites.

> our new query language, HelixQL

But why? Why increase the barrier of entry for your system?

Is there an SVG version of your logo?

I teach any entire lecture on this topic:

https://15445.courses.cs.cmu.edu/fall2024/schedule.html#sep-...

Thomas is the greatest database systems researcher in the world. He is an incredible human.