News for hackers. Stuff that matters.
Re:PgDog is funded and coming to a database near you by Ozzie_osman on Wednesday June 10, 2026 @11:00AM (#48477382)

  We sharded over 20 TB that we know about.
This is probably a typo, right? 20TB isn't that big. I would imagine they've sharded a lot more than that
Re:PgDog is funded and coming to a database near you by rbranson on Wednesday June 10, 2026 @11:35AM (#48477967)
You are correct. As a point of comparison: almost ten years ago at Segment we had a single Aurora PostgreSQL instance with ~50T of data, it was used to index potential identity data in a much larger corpus of files stored in S3.
Re:PgDog is funded and coming to a database near you by GiorgioG on Wednesday June 10, 2026 @11:13AM (#48477598)
For a vast majority of use cases 20TB is positively enormous.
Re:PgDog is funded and coming to a database near you by mplanchard on Wednesday June 10, 2026 @11:47AM (#48478147)
RDS caps out at 64 TB unless you use Aurora, so 20 TB is totally manageable without sharding.
Re:PgDog is funded and coming to a database near you by jeltz on Wednesday June 10, 2026 @11:46AM (#48478132)
Yes. But for most workloads it is not much for PostgreSQL. You often will not have to shard at all.
Re:PgDog is funded and coming to a database near you by returningfory2 on Wednesday June 10, 2026 @11:25AM (#48477771)
This product is for Postgres deployments that are so large they need to be sharded. For these use cases, I think 20TB is about normal.
Re:PgDog is funded and coming to a database near you by happyopossum on Wednesday June 10, 2026 @11:22AM (#48477722)
Sure, but 20TB in “the only database you need” is mere hours or minutes worth of data for many workflows.
Re:PgDog is funded and coming to a database near you by tingletech on Wednesday June 10, 2026 @11:18AM (#48477666)
that article seems to suggest 20TB total over the dozen deployments in prod.
Re:PgDog is funded and coming to a database near you by singron on Wednesday June 10, 2026 @12:41PM (#48478957)
If your working set is 20 TB, then it's pretty big. Each database has its own mix of hot/cold data, so it's impossible to compare without more information. A better measure might be IOPS. RDS has fairly low maximum IOPS unless you spend a lot more for provisioned IOPS or use Aurora.
Re:PgDog is funded and coming to a database near you by dujuku on Wednesday June 10, 2026 @04:14PM (#48481994)
If you think 20TB "isn't that big" I want to know what size of DBs you're working with 0_0
Re:PgDog is funded and coming to a database near you by inigyou on Wednesday June 10, 2026 @09:12PM (#48485054)
It's big but it's not so big it wouldn't fit on SSD on one particularly beefy server (two for redundancy). Sharding this would be more about the transaction rate. Actually, sharding would always be about the transaction rate.
Re:PgDog is funded and coming to a database near you by ComputerGuru on Thursday June 11, 2026 @11:09AM (#48491429)
It doesn’t even remotely need to fit on one SSD with logical volume management (or RAID).
Re:PgDog is funded and coming to a database near you by Ozzie_osman on Thursday June 11, 2026 @01:10AM (#48486455)
I mean yes, for a single DB it's large, but if you're thinking about sharding you're probably in the tens of TBs, and if you're a company offering sharding you've prob sharded larger workloads.
Re:PgDog is funded and coming to a database near you by ubercore on Thursday June 11, 2026 @04:56AM (#48487946)
It's really not that big for a postgres db in a lot of places, honestly.