riccomini
Distributed Compaction in SlateDb
Something Is Happening
MCP Server Could Have Been a JSON File
Show HN: ZeroFS: The S3FS that does not suck
Kafka: The End of the Beginning
Re:Everything You Need to Know About Incremental View Maintenance
on Friday April 18, 2025 @03:03PM (#43730963)
DD supports lattices that allow it to compute at multiple points in time simultaneously. As I understand it, DBSP limits time to one diff at a time. Lalith can correct me if I’m off base on this. :)
Everything You Need to Know About Incremental View Maintenance
The Bond Market at the 5% Threshold
AI IDEs Need Moats
Comparing Apache, CNCF, and Commonhaus
Show HN: A static site generator toolkit written in Python
S3 Is the New SFTP
Slatedb/Clean_slate.md
Virtual Machines Are Getting Better
Stop embedding RocksDB in your Stream Processor
The New Era of Flexible Infrastructure Deployment
Modular Monoliths Are a Good Idea
Re:Show HN: SlateDB – An embedded storage engine built on object storage
on Thursday August 15, 2024 @01:14PM (#41258251)
Great question! We started out with the design you described--WAL as L0. But we found that there's a bit of a tension between wanting to have L0 SSTs be larger (and having fewer of them) to reduce metadata size, while we wanted to keep WAL SSTs small and frequent (to reduce async/await latency).
Basically, we wanted to have WAL writes go on the order of milliseconds, but we wanted L0 SSTs to be larger since they actually service reads.
The architecture page has more detail if you haven't found it yet:
Show HN: SlateDB – An embedded storage engine built on object storage
https://github.com/slatedb/slatedb
SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API...