Re:Catena: A time series storage engine written in Go
by misframer on Saturday March 07, 2015 @09:51PM
(#9164283)
I like Bolt too. It was my first choice :). Using it for read-only partitions sounds like a great idea. I think the current implementation looks like a tree if you "squint" a bit. There's a hierarchy of metrics and sources, and the arrays of points are like the leaf nodes.
You're very correct about B+trees being efficient with sequential reads. We use MySQL with InnoDB for all of our time series where I work[0]. I started this way just to get a good understanding of what works and what doesn't. I think I'm on track to implementing something like a B+tree by using smaller extents of points.
[0] http://www.slideshare.net/vividcortex/vividcortex-building-a...
Also, I'm only talking about using Bolt as the on-disk partition store, ie the read-only partitions. You'd only write them when flushing the in-mem partitions, so no need for concurrent writers.
> tree-based systems aren't very good at doing large, sequential reads.
It all depends on the implementation you've chosen. B+trees exist exactly for efficient sequential reads, and Bolt uses one. (Take a look at this [0]: 40 ns/op per k/v, and putting the database in memory doesn't even improve it).
Trees also allow you to search for specific timestamps so you can
[0] https://paulosuzart.github.io/blog/2014/07/07/going-back-to-...