News for hackers. Stuff that matters.

A comment on How Figma's databases team lived to tell the scale. Parent comment

Re:How Figma's databases team lived to tell the scale by dijit on Thursday March 14, 2024 @02:49PM (#39707557)
I've had CitusDB running across 68 bare metal machines (40 vCPU, 768GiB ram, 20TiB of storage each + 40GiB network links) and it ran decently well.

Not sure what your definition of massive is, I think Spanner would easily beat it.

Also, it's very use-case dependent, you can't "just use" Citus for everything, it's not quite as flexible as a bog-standard pgsql install due to the way it's sharding, you have to be a tad more careful with your data model.

[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by skunkworker on Thursday March 14, 2024 @04:52PM (#39708831)
What is your definition of "decently well", and is your primary cluster (without replicas) above 1PB?
[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by simonw on Thursday March 14, 2024 @05:37PM (#39709269)
They said 20TiB * 68, which I think is 1.5PB.
[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by skunkworker on Friday March 15, 2024 @05:09PM (#39720735)
That could be all of the nodes, or just the primaries without replicas.
[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by VHRanger on Thursday March 14, 2024 @09:57PM (#39711320)
Is there a reason there's comparatively little storage in your machines in relation to RAM or even CPUs?

Do your machines do compute heavy loads or something?

For a DB I'd expect a lot more storage per node

[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by dijit on Friday March 15, 2024 @12:10AM (#39711992)
NVMe SSDs aren't so large unfortunately.

a 1U server has capacity for 8 drives, we used 2 slots for the OS (RAID1), 2 slots for the WAL volume (2 slots) leaving only 4 slots in RAID10.

So I'm already cheating a little and claiming WAL storage was part of total storage.

[ Parent | on HN ]
Re:How Figma's databases team lived to tell the scale by VHRanger on Friday March 15, 2024 @08:50AM (#39714937)
Shouldn't you try to get something with PCIe bifurcation in this case?

I doubt you're saturating the PCIe bus bandwidth on any of them?

I imagine your DB is extremely high performance, though!

[ Parent | on HN ]