News for hackers. Stuff that matters.
Re:Features I'd Like in PostgreSQL by whitepoplar on Saturday January 28, 2023 @03:41PM (#34561602)
I'd love to see the high-availability story improve in core, as well as horizontal scalability built into core.
Re:Features I'd Like in PostgreSQL by morrbo on Saturday January 28, 2023 @04:06PM (#34561875)
Never agreed with something so much more in my life. Points 1-3 in particular. I understand their logic of not wanting to "pollute" the core products with these features.... but it's sort of past the point now where you're starting to expect this stuff in a modern database product (instead of add-ons, hacks, and outdated blog posts)
Re:Features I'd Like in PostgreSQL by seedless-sensat on Saturday January 28, 2023 @04:24PM (#34562071)
Points 1-3 exactly describe my pains operating Postgres at scale
Re:Features I'd Like in PostgreSQL by threeseed on Saturday January 28, 2023 @04:27PM (#34562105)
It is kind of ridiculous that the first three haven't been sorted by now.

And over time it will increasingly relegate PostgreSQL to being for development only with production use being handled by wire-compatible databases e.g. Aurora.

Re:Features I'd Like in PostgreSQL by ahachete on Saturday January 28, 2023 @04:29PM (#34562126)
> 2) Horizontal scalability without having to resort to an extension like Citus.

Just curious, what would save you having the solution in-core? Installation, sure, but that's a one-off possibly in your deployment code. "CREATE EXTENSION citus" and add that to postgresql.conf? Sure, but not too much work for me. The rest (commands to actually create the nodes, do the sharding itself) are something I cannot imagine being different or simpler if with an in-core solution.

What am I missing?

Re:Features I'd Like in PostgreSQL by threeseed on Saturday January 28, 2023 @04:32PM (#34562162)
It means that when you upgrade PostgreSQL you don't have to worry if the extensions are compatible and have been fully tested not just for functionality but security etc as well.

And most importantly it means you don't have to worry if that extension will move to a freemium model which (a) often has important features out of your price range and (b) is generally unacceptable in enterprise environments.

Re:Features I'd Like in PostgreSQL by ahachete on Saturday January 28, 2023 @07:27PM (#34563532)
Right, those are compulsory steps for every upgrade.

Yet in the particular case of Citus, history (so far) has shown a) that they update the extension regularly and fast, so by the time you want to upgrade to a newer major version you already have Citus updated too; b) they are going exactly in the opposite direction of "freemium", they actually open sourced even the previous proprietary bits; c) as OSS, it can always be forked and if one day closed source, being such an important project, it would be definitely forked.

(I don't have any stakes on Citus)

Re:Features I'd Like in PostgreSQL by Keyframe on Sunday January 29, 2023 @12:23AM (#34565208)
Citus is Microsoft for quite awhile now. Take what you will from that.
Re:Features I'd Like in PostgreSQL by threeseed on Sunday January 29, 2023 @01:11AM (#34565440)
The "you can always fork Citus" argument is not workable.

Scaling databases is an expert skill that is far out of reach for almost everyone.

Far easier just to pick another database that includes HA and horizontal scalability out of the box.

Re:Features I'd Like in PostgreSQL by ahachete on Sunday January 29, 2023 @09:08AM (#34568203)
Sharding databases is not such a dark, magic art. Citus relies on Postgres for many key features; and Citus does the rest. It's already quite "feature complete".

If Citus would become proprietary overnight, my main concerns of maintaining a fork would be around the codebase and the language expertise more than the sharding concepts.

Note that sharding is different from a purely distributed database. The latter is an entirely different class (and more complex system).

Re:Features I'd Like in PostgreSQL by threeseed on Sunday January 29, 2023 @10:42AM (#34569066)
The more likely scenario is that Citus simply disappears entirely and becomes an internal feature of Azure.
Re:Features I'd Like in PostgreSQL by contrahax on Saturday January 28, 2023 @09:23PM (#34564239)
You can’t use those extensions on almost any cloud like AWS, GCP, Aiven, etc. - I think Azure is the only one offering a citus product because they acquired them. Also extension updates are a major pain point in hosted DBs - always lagging behind. Having some solution in core would resolve this, even if it was just bundling some best in class extensions out of the box to bypass these cloud providers very selective extension support.
Re:Features I'd Like in PostgreSQL by LtWorf on Saturday January 28, 2023 @04:32PM (#34562164)
What would the proprietary vendors sell then?
Re:Features I'd Like in PostgreSQL by viraptor on Saturday January 28, 2023 @06:17PM (#34563024)
What do you mean by query pinning? Something like query cache where the results are kept around explicitly?
Re:Features I'd Like in PostgreSQL by whitepoplar on Saturday January 28, 2023 @07:20PM (#34563473)
Sorry, query plan pinning.
Re:Features I'd Like in PostgreSQL by gavinray on Saturday January 28, 2023 @07:31PM (#34563568)
It's not baked-in, but Andy Pavlo has an ML-based tuning tool called "Ottertune" for Postgres:

I think it's free for a single DB

https://ottertune.com/

Re:Features I'd Like in PostgreSQL by tbillington on Saturday January 28, 2023 @08:56PM (#34564086)
Only for instances on aws though.
Re:Features I'd Like in PostgreSQL by ies7 on Saturday January 28, 2023 @07:55PM (#34563733)
We've used postgres for so long that number 1 & 3 feels just automatic for our teams to setup and don't ask why they're not built-in in postgres.
Re:Features I'd Like in PostgreSQL by ilyt on Saturday January 28, 2023 @09:10PM (#34564163)
Ye at the point where you need it (even if you decide to do it on your own and not use some cloud solution) it's one off time expense
Re:Features I'd Like in PostgreSQL by efxhoy on Sunday January 29, 2023 @05:03AM (#34566622)
Which tools do you use for 1 and 3?
Re:Features I'd Like in PostgreSQL by ksec on Saturday January 28, 2023 @10:45PM (#34564684)
It is funny to see this because every time these reasons for people continue to use MySQL are listed, PostgreSQL folks will be quick to reply most of these are non-issues.
Re:Features I'd Like in PostgreSQL by KyeRussell on Sunday January 29, 2023 @03:19AM (#34566016)
This sort of tribal framing doesn’t do anything to help the situation. I’m immediately willing to believe that the “Postgres folks” that are saying that, genuinely have not found these points to be issues, which doesn’t mean that OP doesn’t.
Re:Features I'd Like in PostgreSQL by qazxcvbnm on Saturday January 28, 2023 @10:57PM (#34564751)
For 4) I've heard there's this available https://github.com/DrPostgres/pg_plan_guarantee
Re:Features I'd Like in PostgreSQL by avinassh on Saturday January 28, 2023 @11:56PM (#34565091)
How does Citus provide horizontal scaling? How does it work?
Re:Features I'd Like in PostgreSQL by jeltz on Sunday January 29, 2023 @07:33AM (#34567510)
Sharding.
Re:Features I'd Like in PostgreSQL by zX41ZdbW on Sunday January 29, 2023 @05:58PM (#34573683)
I found Citus implementation quite weak among analytic databases. It does not fare well even among Postgres-based systems, like Greenplum [1].

[1] https://benchmark.clickhouse.com/

Re:Features I'd Like in PostgreSQL by EamonnMR on Sunday January 29, 2023 @12:53AM (#34565329)
Big second on horizontal scaling. Amazon Redshift is a great example of what's possible, but it also has enough problems that you can't really use it as a primary application database.
Re:Features I'd Like in PostgreSQL by zX41ZdbW on Sunday January 29, 2023 @05:56PM (#34573668)
Redshift is an OLAP database, much like ClickHouse. For OLAP databases, horizontal scalability is a must - due to generally larger data volumes.

While Postgres is OLTP. For OLTP databases achieving horizontal scalability require a more sophisticated approach to distributed consensus, like in CockroachDB or Spanner.

Re:Features I'd Like in PostgreSQL by _448 on Sunday January 29, 2023 @03:30AM (#34566071)
> Horizontal scalability without having to resort to an extension like Citus.

Citrus also has one glaring problem: single point of failure due to a single monitor node.

Postgres would blow every other database away if it had HA, multi-write nodes and fault tolerant feature similar to FoundationDB or TiDB.

Re:Features I'd Like in PostgreSQL by smw on Sunday January 29, 2023 @04:36AM (#34566450)
Isn't that what CockroachDB is?
Re:Features I'd Like in PostgreSQL by _448 on Sunday January 29, 2023 @12:40PM (#34570490)
It suffers from licensing spaghetti.
Re:Features I'd Like in PostgreSQL by Existenceblinks on Sunday January 29, 2023 @05:05AM (#34566638)
I would love 6) unique constraints across all partitions
Re:Features I'd Like in PostgreSQL by boiler_up800 on Sunday January 29, 2023 @08:00AM (#34567688)
Yugabyte delivers most of these. It’s also the “purest” implantation. I expect it to be the winner over time of the Postgres scaling wars.