News for hackers. Stuff that matters.
Re:Membrane, Media Framework for Elixir by dlachausse on Tuesday May 13, 2025 @01:35PM (#43975510)
The BEAM even includes a database - Mnesia...

https://www.erlang.org/doc/apps/mnesia/mnesia.html

Re:Membrane, Media Framework for Elixir by jerf on Tuesday May 13, 2025 @02:32PM (#43976131)
Mnesia is not a database by any modern definition of the term and it should generally be avoided. It is at least 4 if not 5 orders of magnitude away from "being able to run Twitch". That is, yes, I'm serious, if you tried to run a Twitch clone "but 10,000x smaller" I would still expect Mnesia to completely fall over.
Re:Membrane, Media Framework for Elixir by toast0 on Tuesday May 13, 2025 @03:17PM (#43976553)
Mnesia worked well enough for us at WhatsApp while I was there; although we didn't use it to store messages; long term message storage is on the end points (generally sqlite), messages in transit (offline) were stored in a file per user with the import/export written in C IIRC. We did add redundant in memory storage of messages in transit; but I don't remember the storage there; may have just been ets.

We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables.

We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory.

Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific.

If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.

Re:Membrane, Media Framework for Elixir by victorbjorklund on Wednesday May 14, 2025 @08:02AM (#43983476)
That is not true. I dont use Mnesia because it is so different and thus harder for me than sql. But it is used. Klarna uses it. They are a pretty big company. Would I recommend it for most? No, but I also would not recommend cassandra and a bunch of other databases for most apps
Re:Membrane, Media Framework for Elixir by schultzer on Tuesday May 13, 2025 @02:42PM (#43976217)
mnesia is great, and you can get very far before you would make the jump to anything else. And it can be way faster then any other database. For obvious reasons.

Although it would be great if it spoke SQL, maybe one day it will: https://github.com/elixir-dbvisor/sql since we can already pass it and get the AST.

Re:Membrane, Media Framework for Elixir by troupo on Tuesday May 13, 2025 @03:31PM (#43976706)
And when you do have to jump off, you'll be screwed. Because the jump off is usually from in-memory Mnesia with guaranteed sub-millisecond responses to a proper database with at least a magnitude higher latency. And you realize that your app is completely dependent on sub-millisecond responses :)
Re:Membrane, Media Framework for Elixir by schultzer on Tuesday May 13, 2025 @04:37PM (#43977433)
Low latency is addictive. And by the time when you think you might need to change you’re probably become such a skilled engineer that you realize that everybody that shit on mnesia does not know what they talk about. :)
Re:Membrane, Media Framework for Elixir by throwawaymaths on Tuesday May 13, 2025 @02:12PM (#43975927)
Honestly I think mnesia is one of those "don't use it unless you know what you're doing" things. Just use postgres.