News for hackers. Stuff that matters.

A comment on Moving a billion Postgres rows on a $100 budget.

Re:Moving a billion Postgres rows on a $100 budget by nojvek on Wednesday February 21, 2024 @06:30PM (#39461147)
Postgres seriously needs a columnstore backed table instead of just a rowstore.

MSSQL has this and it is magic. SingleStore has it, and it is wonderful.

I'm willing to give a bounty of $1000 to whoever adds that into main postgres tree.

Snowflake is great as a warehouse. it's latency is shit when it comes to fast lookups and aggregates. If you can tolerate >1s api calls, that is fine. It takes forever to insert a few rows in a large table.

If you want a proper live DB, snowflake is a rich man's poor database.

[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by dkatz23238 on Wednesday February 21, 2024 @06:58PM (#39461339)
Parade DB are working on this problem: https://github.com/paradedb/paradedb/tree/dev/pg_analytics
[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by CSSer on Wednesday February 21, 2024 @10:21PM (#39462810)
Know anybody who could do it in a weekend? That’s about what you’re paying for.
[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by speedgoose on Thursday February 22, 2024 @01:09AM (#39463750)
Columnar store PostgreSQL extension exists, here are two but I think I’m missing at least another one:

https://github.com/citusdata/cstore_fdw

https://github.com/hydradatabase/hydra

You can also connect other stores using the foreign data wrappers, like parquet files stored on an object store, duckdb, clickhouse… though the joins aren’t optimised as PostgreSQL would do full scan on the external table when joining.

[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by Loic on Thursday February 22, 2024 @01:58AM (#39464037)
For time series, you have the TimescaleDB extension.

https://www.timescale.com/

[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by ddorian43 on Thursday February 22, 2024 @02:34AM (#39464276)
> I'm willing to give a bounty of $1000 to whoever adds that into main postgres tree.

This is the most disrespectful thing I've read on HN.

[ Parent | on HN ]
Re:Moving a billion Postgres rows on a $100 budget by nojvek on Thursday February 22, 2024 @01:51PM (#39471353)
I'm sorry my intent came off in the wrong way. I am not super wealthy.
[ Parent | on HN ]