Indices are one the first things you learn in any decent DB course, so you don’t need a PhD

But if BigTable just solves the problem it seems the way to go

PostGres is popular for a reason, it’s ACID unlike BigTable

You may run into these problems later on, you may not

I love Postgres and use it a lot but not for analytics.

ACID is good and it has an implementation cost (vacuum for example). When doing analytics you do not care for ACID

I have 50 columns in a table with 10B records in a partitioned table. People can search using any combination. There are a huge number of possible composite indexes, so we don't have any, but we have every column indexed. Most queries take a minute or two, and we also experienced a sudden drop in performance even for queries that used to work OK. Not sure what to do next to improve query speed.

This is an interesting workload. We've heard similar pains for fast filtering on many large column tables.

For this workload, having a columnstore version of your table will help.

DM us: https://join.slack.com/t/mooncakelabs/shared_invite/zt-2sepj.... We can help.

Yea this is indeed a repeated pattern we saw people requesting (filter on many columns) and we are trying to solve with pg_mooncake. If you are interested, feel free to join mooncake-devs.slack.com to chat more about your use case.