hubertzhang

I'm the author... Codex is co-author but the blog “Migrate Your 300-Node Redis Cluster to Just 3 Lavik Nodes” is reviewed by myself.

Yes, I think it's Hackernews admin help to promote this one.

Yes, we compared Lavik with DragonflyDB on flash

https://lavik.dev/en/compare/lavik-vs-dragonfly/

Do you mean the use case of a 300-node Redis Cluster?

It’s very common. Since Redis typically runs with 2 vCores per node, a 300-node cluster can only store about 2.4 TB of data.

It’s common in human-centric feature stores and profile stores. But it’s even more common in agent profile stores, since agent profiles are a much larger category.

Why keep buying servers for RAM? Lavik combines NVMe SSD capacity with throughput above Redis on DRAM. Explore the architecture, benchmarks & economics of a 300-to-3 consolidation.

Github: https://github.com/microsoft/garnet

Competitors: 1. DragonflyDB: https://github.com/dragonflydb/dragonfly

2. Lavik: https://github.com/eloqdata/lavik

DragonflyDB is famous for 4 millions QPS on single node. While Lavik focus on optimizing Redis workload on NVMe SSD.

Snowflake also uses S3 as primary storage for OLAP workload, what's diff?

we leverage batch write optimization which uses Copy-on-write B-tree variant enables high-throughput batch writes without blocking concurrent reads. MVCC-based design eliminates lock contention and provides predictable write amplification.

Most Redis alternatives that use disk for persistence struggle with tail latency (P9999) due to background maintenance or OS filesystem overhead. We built EloqKV on a custom storage engine, EloqStore, to solve this.

Key Architectural Choices:

- Custom B-tree Variant: Unlike LSM-trees used in many disk-backed stores, our B-tree variant avoids the "compaction stalls" that typically cause high tail latency during heavy writes.

- Coroutines & io_uring: We leverage io_uring for asynchronous I/O and use coroutines to manage thousands of concurrent I/O requests without the context-switching overhead.

- Object Storage Integration (optional): EloqStore uses object storage as the primary persistent layer, with NVMe acting as a high-speed cache/tier, providing durability without sacrificing speed.

We’ve reached a point where we can provide predictable P99.99 latency even when the working set is primarily on NVMe. We’d love to answer any questions about the storage internals or our benchmarking process.

My reflections on the AWS us-east-1 outage.

Yesterday I posted a Show HN introducing our MongoDB-compatible database EloqDoc (https://news.ycombinator.com/item?id=45634638 ). Today’s AWS us-east-1 outage was a strong reminder of why relying on local NVMe as primary database storage is risky—and why OLTP databases should consider object storage as the durable foundation instead.

I cannot pull images from docker hub.

Like AWS, json is OK for me

A recent hacker-news argues: “Redis is fast — I’ll cache in Postgres.” The author benchmarks a simple API and concludes that although Redis is faster, Postgres is “fast enough,” and removing another component can be worth it for many projects.(https://news.ycombinator.com/item?id=45380699)

I agree with two things:

- Postgres is an excellent, reliable database.

- Fewer moving parts is a win—when the workload fits.

But there’s a third path that many teams overlook: a Redis-compatible database that is durable by default. That’s what we built with EloqKV—Redis protocol + redo log, multi-writer, transactions, persistence, durability, store procedure—so you get database + cache in one engine.

Any thoughts?

That use case aligns perfectly with EloqKV’s capabilities. In pure cache mode, batching multi-key deletions within Lua scripts can significantly reduce latency by minimizing client–server round trips.

Yes, Lua in EloqKV has no such limitations. You can freely read data, generate new keys, and even query those keys within Lua scripts. Underneath, EloqKV’s transaction layer is powered by our data substrate, which provides full ACID guarantees. FYI https://www.eloqdata.com/blog/2025/07/14/technology

Could you share a bit more about your specific use case? That will help me explain how EloqKV can best support it.

I’m Hubert, CTO of EloqData. That’s a great question.

Redis Cluster is often thought of as a distributed database, but in reality it’s not truly distributed. It relies on a smart client to route queries to the correct shard—similar to how mongos works in MongoDB. This design means Redis Cluster cannot perform distributed transactions, and developers often need to use hashtags to manually place related data on the same shard.

EloqKV takes a different approach. It’s a natively distributed database with direct interconnects between nodes. You can connect to any node with a standard Redis client, and still read or write data that physically resides on other nodes. This architecture enables true distributed transactions across shards, fully supporting MULTI/EXEC and Lua scripts without special client logic or manual sharding workarounds.

Some Redis variants focus on persistence—KVrocks, for example, or MemoryDB, which emphasizes durability through redo logs to minimize data loss. However, they are not truly transactional, since they lack fundamental rollback semantics and distributed transaction.

EloqKV, by contrast, is fully transactional. It supports distributed Lua, MULTI/EXEC, and even SQL-style BEGIN/COMMIT/ROLLBACK syntax. This means you get the transactional guarantees of a database with Redis-level read performance. Writes are slightly slower since EloqKV ensures durability, but in return you gain full ACID safety. Most importantly, you no longer need to worry about cache coherence issues between a Redis cache and a separate SQL database—EloqKV unifies them into a single, reliable system.

Does Neki still need sharding key in query, just like Citus?

I’m the co-founder of EloqData. I recently gave a talk on EloqDoc and Pulsar integration at Data Streaming Summit 2025. It was a great event, and Ursa was definitely one of the hottest topics discussed.

I believe object storage is shaping the future architecture of cloud databases. The first big shift happened in the data warehouse space, where we saw the move from Teradata and Greenplum to Snowflake accelerate around 2016. Snowflake’s adoption of object storage as its primary storage layer not only reduced costs but also unlocked true elasticity.

Now, we’re seeing a similar trend in the streaming world. If I recall correctly, Ursa was the first to GA an object-storage–based streaming service, with Kafka(WarpStream) and AutoMQ following afterward.

I also believe the next generation of OLTP databases will use object storage as their main storage layer. This blog post shares some insights into this trend and the unique challenges of implementing object storage correctly for OLTP workloads, which are much more latency-sensitive.

https://www.eloqdata.com/blog/2025/07/16/data-substrate-bene...

Is Meta/X/Google open source? Is Gamil/Outlook open source? All of them gather user's data. They are all "free" software

Our CEO Jeff replied with this passage but seems that his account is too new and got blocked. I repost it here:

You're absolutely right—discussing the CAP theorem is essential for any distributed storage system. We are preparing a detailed blog post to cover the implications of CAP in our system. In brief, our system generally aligns with PC/EC in PACELC terms (https://en.wikipedia.org/wiki/PACELC_theorem), similar to other fully distributed databases like CockroachDB and TiDB. However, this configuration is flexible. For instance, in high-performance cache applications where durability is less critical, our system can be adjusted to favor PA/EL, akin to many NoSQL systems.

Regarding persistence, our Substrate component manages its own logging system while relies on external storage engines for checkpointing. This means that the system only depends on the performance and capacity of the external storage. Our database does not depend on the external storage engine for consistency. For example, DynamoDB offers virtually unlimited capacity and performance (in terms of QPS) in the cloud, and our Data Substrate is agnostic to whether the data is stored as a B-Tree or an LSM tree.

We are currently preparing the software for Jepsen testing prior to its General Availability (GA) release. More technical details will be shared as we move forward, so please stay tuned.

> It superficially sounds like a series of server processes fronting actual database servers, which sounds like another layer of partition vulnerability and points of failure

I agree that introducing a middleware layer on top of databases adds more points of failure. EloqKV avoids this by integrating Compute, Logging, and Storage into a single system. In this setup, storage can be a database like Cassandra, but users will not access it directly; all requests go through EloqKV, which manages ACID transactions. EloqKV is responsible for handling crashes of the TxServer, LogServer, and Storage. You can think of the distributed Storage Engine just as a Disk in traditional DBMS. Obviously its failure will affect the system. But no more than hard disk failure. In fact, in a cloud, all disks (i.e. EBS) are actually distributed storage systems.

This situation is a rethinking of the Redis and MySQL combination, which suffers from similar issues. Both systems can fail independently, resulting in only eventual consistency. EloqKV aims to resolve this problem.

Good question! WAL and TxService(TxMap) are fully decoupled, allowing you to deploy them on the same node or across different machines. If your workload is write-intensive, you might consider a large WAL cluster, which leverages multiple disks to perform fsync operations in parallel. This is particularly important in cloud environments, where local SSDs are ephemeral and cloud-native databases often use EBS for WAL. However, high-performance EBS options like io2 can be costly. EloqKV's decoupled architecture allows you to scale write throughput up to 600K Write OPS on a single TxServer as you increase the number of cheap gp3 disks. Since WAL logs can be truncated after a checkpoint, the required disk size is typically quite small, making gp3 a cost-effective choice.

For more details, please check our scaling disks benchmark report.

https://www.eloqdata.com/blog/2024/08/25/benchmark-txlog#exp...

> For a distributed transaction with distributed storage and multiple nodes how does a transaction get coordinated?

EloqKV is a multi-writer system, similar to many distributed databases (FoundationDB, TiDB, CockroachDB), but we have a set of new innovations on transaction protocols, for example, the entire system operates without a single synchronization point, not even a global sequencer. We drew many inspirations from the Hekaton and the TicToc paper.

Good question! As explained in the linked article, our core technology, Data Substrate, is modular by design and requires a storage layer to handle the actual data (more precisely store the checkpoints). This storage layer can be any data store with a Key-Value interface: it doesn't need to meet the consistency and isolation requirements of a full database.

We chose RocksDB and Cassandra to showcase two different approaches. RocksDB offers efficient local storage, but if there's a disk hardware failure, the data can be lost. Cassandra, on the other hand, ensures high availability by replicating data, making it resilient to disk failures. We also support cloud storage options like DynamoDB and BigTable in our cloud offerings.

The current release is a prebuilt binary, giving users flexibility in how they use the software. The open-source model is still being decided, but a license file is included in the downloaded tarball.