Re:Distributed SQLite with Elixir
by nesarkvechnep on Thursday February 16, 2023 @02:35AM
(#34816050)
You can’t get it in C# or Java. The BEAM virtual machine is build upon the idea of lightweight processes. You might get an imitation in other mainstream places but not the real thing. Take a look at the Actor Model if you haven’t heard of it.
Re:Distributed SQLite with Elixir
by weatherlight on Friday February 17, 2023 @11:04AM
(#34836269)
Erlang uses a preemptive scheduler for process scheduling, allowing for parallelization on multiple CPUs, while Scala distinguishes between thread-based and event-based actors. Thread-based actors can perform long-running computations without hindering other actors, but use more memory and scheduling overhead. Event-based actors are more lightweight but not suitable for parallelism as they execute on the same thread.
Re:Distributed SQLite with Elixir
by nl on Wednesday February 15, 2023 @11:58PM
(#34815236)
This is such a weird post.
They build a distributed counter that is consistent across a cluster using the Elixir message passing thing, but there's this whole thing about using the Fly Distributed SQLite thing as well.
But if they wanted to build a distributed counter they could choose to use Postgres at the first step and then it will just work.
Re:Distributed SQLite with Elixir
by ammmir on Thursday February 16, 2023 @12:08AM
(#34815295)
Manually replicating writes to all nodes seems pretty fragile. At some point you're going to run into classic distributed systems issues, like netsplits, and reconciling the latest count. Maybe a CRDT implementation would avoid these issues alltogether?
But it sounds like LiteFS (with a single writer?) might be a pretty nice setup if it handles replication for you, as you probably don't want to build a Raft or Paxos system yourself.
Re:Distributed SQLite with Elixir
by simonw on Thursday February 16, 2023 @01:33AM
(#34815754)
Litestream is different from LiteFS: same author, different projects.
LiteFS provides replication with the ability to serve read traffic from those replicas. Litestream provides disaster recovery by streaming a constant backup to an S3 bucket (or similar).
What feature is this and does it not really exist in C# or Java?
You can get this in other languages with the help of middleware like RabbitMQ or Kafka, but not many languages have it built in.
Could you further elaborate on this?
I am familiar with Actor Model. I had work related to Akka in Scala that works on the principles of Actor Model.
They build a distributed counter that is consistent across a cluster using the Elixir message passing thing, but there's this whole thing about using the Fly Distributed SQLite thing as well.
But if they wanted to build a distributed counter they could choose to use Postgres at the first step and then it will just work.
I like the design in many ways, but still need to find something noncritical to test the solution on before clearing it for production. =)
However, there are some designs one should avoid... like auto increment indices etc. =)
But it sounds like LiteFS (with a single writer?) might be a pretty nice setup if it handles replication for you, as you probably don't want to build a Raft or Paxos system yourself.
LiteFS provides replication with the ability to serve read traffic from those replicas. Litestream provides disaster recovery by streaming a constant backup to an S3 bucket (or similar).