News for hackers. Stuff that matters.

A comment on OpenAI: Scaling PostgreSQL to the Next Level. Parent comment

Re:OpenAI: Scaling PostgreSQL to the Next Level by samwillis on Friday May 23, 2025 @09:41AM (#44072806)
The message of the talk was very much that you can scale to massive throughput without having to shard and having only a single master.

Of course they considered it, but the tradeoffs didn't match what they wanted to do - plus they found you could scale to this level without sharding.

[ Parent | on HN ]
Re:OpenAI: Scaling PostgreSQL to the Next Level by vanviegen on Friday May 23, 2025 @10:24AM (#44073200)
The talk seems to be mostly about all the limitations and workarounds they've had to deal with, because they choose not to shard. Apparently, they have a policy of adding no new functionality to the database, which presumably means additional separate database services being setup for each new feature. That sounds a lot like accumulating tech debt very rapidly, just because sharding is not on the table, for whatever reason.
[ Parent | on HN ]
Re:OpenAI: Scaling PostgreSQL to the Next Level by bhouston on Friday May 23, 2025 @01:08PM (#44074565)
Yeah, when they mentioned that they couldn't put any more services on their main DB because of this issue I did a facepalm. They are building out explicit tech debt now because they are not sharding.
[ Parent | on HN ]