Lessons learned scaling a PostgreSQL database to 1.2bn records/month

Re:Lessons learned scaling a PostgreSQL database to 1.2bn re... by merb on Tuesday January 29, 2019 @09:58AM (#19026782)

What's even more strange is that the PostgreSQL of GCloud is relatively simple. It's just a https://cloud.google.com/compute/docs/disks/#repds replicated disk. Probably inside a K8s Cluster.

I mean setting this up is extremly simple, in a High availability setting on K8s. So I would guess that a company in the size of Google can actually find out how to add a way to run pg_upgrade to upgrade a cluster (inject something into the image that runs the task....). Even on AWS it will run pg_upgrade with a downtime (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...).

[ Parent | on HN ]

Re:Lessons learned scaling a PostgreSQL database to 1.2bn re... by paulddraper on Tuesday January 29, 2019 @07:51PM (#19031526)

Unless you make the ill-advised choice to use AWS Aurora :(

We did that and only then found out that despite documentation suggesting the contrary [1], there is no upgrade path for major versions of AWS Aurora PostgreSQL. You're stuck with whatever you had when you started. (Unless you want to do a pg_backup/pg_restore which could take weeks.)

[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...

[ Parent | on HN ]