Re:Open Sourcing Dicer: Databricks' Auto-Sharder
by vivek-jain on Tuesday January 13, 2026 @03:11PM
(#46607170)
Sharded in-memory caching turns out to be rather useful at scale :)
Some of the key examples highlighted on our blog are Unity Catalog, which is essentially the metadata layer for Databricks, our Query Orchestration Engine, and our distributed remote cache. See the blog post for more!
Some of the key examples highlighted on our blog are Unity Catalog, which is essentially the metadata layer for Databricks, our Query Orchestration Engine, and our distributed remote cache. See the blog post for more!