Re:How Figma's databases team lived to tell the scale
by themoonisachees on Thursday March 14, 2024 @07:23PM
(#39710187)
An advantage worth noting is that having actually separated databases means you physically can't make these expensive operations, so a junior dev can't write incredibly inefficient code that would bring down your entire infra.
Re:How Figma's databases team lived to tell the scale
by ummonk on Thursday March 14, 2024 @11:35PM
(#39711818)
Also makes it a lot harder for devs to do some footgun and leak data across domains.
Re:How Figma's databases team lived to tell the scale
by aledalgrande on Thursday March 14, 2024 @11:54PM
(#39711912)
Bring down the infra or foot you a 6 figures bill at the end of the month
Re:How Figma's databases team lived to tell the scale
by shalabhc on Thursday March 14, 2024 @10:20PM
(#39711431)
"Lots of Little Databases" reminded me of https://www.actordb.com/ which does lots of server-side sqlite instances, but the project now looks defunct.
I've been calling it the Lots of Little Databases model vs the Globe Spanning Gorilla.
Like the Spanner paper points out, even if your distributed database semantically appears like a single giant instance, in practice performance means developers avoid using distributed joins, etc, because these can lead to shuffling very large amounts of intermediate results across the network. So the illusion of being on a single giant machine ends up leaking through the reality, and people end up writing workarounds for distributed joins like async materialization.
If we give up the single machine illusion we get a lot of simplification, at the cost of features devs were unlikely to use anyhow. I see having consistent distributed commit but without cross shard joins as a really interesting alternative.
And besides scalability I like the extra security rope of fine grained partitioning from the start.
I'll write a blog post along these lines if I get anything worthwhile done.