Re:An Overview of Distributed PostgreSQL Architectures
by mistrial9 on Monday January 08, 2024 @09:09PM
(#38921329)
second yes to that - postgresql warm with plenty of RAM can do some fancy things and return an answer sub-millisecond too
cache is King
Re:An Overview of Distributed PostgreSQL Architectures
by franckpachot on Tuesday January 09, 2024 @08:25AM
(#38925888)
but large cache is expensive in the cloud and you cannot scale up/down without downtime
Re:An Overview of Distributed PostgreSQL Architectures
by datadrivenangel on Tuesday January 09, 2024 @10:25AM
(#38927179)
4TB of ram is only $71 per hour on AWS RDS. If you're at planetary scale that's not bad.
Re:An Overview of Distributed PostgreSQL Architectures
by franckpachot on Tuesday January 09, 2024 @01:12AM
(#38922976)
The "disappointing" benchmark mentioned in the article is a shame for GigaOm who published it and for Microsoft who paid for it. They compare Citus with no HA to CockroachDB and YugabyteDB with replication factor 3 Multi-AZ, resilient to data center failure. And they run Citus on 16 cores (=32 vCPU) and the others on 16 vCPU.
But your point about "beefy machine" shows the real advantages of Distributed SQL. PostgreSQL and Citus needs downtime to save cost if you don't need that beefy machine all days all year. Scale up and down is downtime, as well as upgrades. Distributed SQL offers elasticity (no downtime to resize the cluster) and high availability (no downtime on failure or maintenance)
Re:An Overview of Distributed PostgreSQL Architectures
by AdamProut on Tuesday January 09, 2024 @11:30AM
(#38928026)
RE: "Distributed SQL offers elasticity (no downtime resize"). I'm not sure this is as much of an advantage of distributed databases vs single host databases anymore. Some of the tech to move virtual machines between machines quickly (without dropping TCP connections) is pretty neat. Neon has a blog post about it here[1]. Aurora Serverless V2 does the same thing (but I can't find a detailed technical blog post talking about how it works). Your still limited by "one big host" but its no longer as big of a deal to scale your compute up/down within that limit.
Single node Postgres (with a beefy machine) can definitely manage in the 100k transactions per second. When you're pushing the high 100k into millions read replicas is a common approach.
When we're talking transactions, question of is it simply basic queries, bigger aggregations, and is it writes or reads. Writes if you can manage to do any form of multi-line insert or batching with copy you can push basic Postgres really far... From some benchmarks Citus as mentioned can hit millions of records per second safely with those approaches, and even without Citus can get pretty high write throughput.