Re:Membrane, Media Framework for Elixir
by toast0 on Tuesday May 13, 2025 @03:17PM
(#43976553)
Mnesia worked well enough for us at WhatsApp while I was there; although we didn't use it to store messages; long term message storage is on the end points (generally sqlite), messages in transit (offline) were stored in a file per user with the import/export written in C IIRC. We did add redundant in memory storage of messages in transit; but I don't remember the storage there; may have just been ets.
We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables.
We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory.
Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific.
If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.
We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables.
We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory.
Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific.
If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.