gawry

Yes! RocksDB is incredibly powerful with proper setup and key schemas.

Following on that project I might build other things like a stream/queue broker, which will probably fit decently with RocksDB as well

Part 3 of my distributed database series. This one covers the storage engine decision - the foundation everything else sits on.

Main topics: how key encoding lets a single ordered KV store emulate document/graph/time-series models, LSM-tree vs B-tree trade-offs, and the benchmark that killed my pure Elixir dreams.

I wanted to use CubDB (pure Elixir, no NIF risks, easy debugging). The benchmarks said otherwise: RocksDB was 177x faster on writes and used 26,000x less memory during batch operations. For a distributed database, that gap is insurmountable.

The post also covers living with NIFs in Elixir - they bypass the BEAM scheduler, so a crash kills your VM instead of just a process. You architect around it: shard isolation, replication, aggressive monitoring.

Also discussed: RocksDB column families (underrated feature for multi-model storage), write amplification as the LSM-tree tax, and why this approach handles time-series data but won't compete with columnar engines like ClickHouse for pure analytics.

Next post will cover Raft consensus for metadata and how the CP metadata plane coordinates with the AP data plane.

Happy to discuss storage engine choices, NIF risk mitigation, or whether the CubDB benchmarks surprised anyone else who's used it.

Part 2 of documenting my distributed database project. This post covers why Elixir's BEAM VM is well-suited for building distributed databases: OTP supervision trees, native clustering, lightweight processes, and the ecosystem (libcluster, Phoenix.PubSub, etc.).

I'm building a hybrid CP/AP system - Raft consensus for metadata (schemas, ring state) while keeping data operations AP for high availability. Decided to build custom gossip for ring distribution instead of using Horde to get better metadata control and prepare for multi-AZ deployments.

Essentially rebuilding Riak Core with better metadata handling. Currently using :rpc but abstracting transport for future flexibility. May need Partisan for clusters >100 nodes but starting simple.

Happy to discuss architecture trade-offs, the hybrid consistency model, or whether rebuilding Riak Core is crazy vs. just forking it.

It is getting types but as an after thought, so i still don't like that much how that's done. Better than not having though

I used to build webapps, and currently working on building a database

I would like to see that too. I have been using Elixir for projects with good results.

But I like Gleam's syntax and the idea of having compile time type checks.

I'm documenting my journey building a distributed database using Elixir/OTP and RocksDB. Not trying to replace Postgres, but exploring whether Elixir's concurrency primitives and supervision trees offer a fundamentally different approach to distributed storage. This first post covers the known hard problems: CAP theorem trade-offs, conflict resolution, node failures, network partitions, and why these challenges are actually interesting to work through. Happy to discuss any of the technical decisions or answer questions about the approach so far.

A nice place to start might be the CFA study guide

https://www.cfainstitute.org/programs/cfaprogram/courseofstu...

They make a comparison in their Docs. http://docs.telepat.io/#comparison

Doesn't unix socket commands work without dropping connections?

http://cbonte.github.io/haproxy-dconv/configuration-1.5.html...