Re:Beringei: A high-performance time series storage engine
by jdonaldson on Saturday February 04, 2017 @04:17AM
(#13566345)
They refer to it as "delta of delta", which implies the encoding handles things like acceleration and momentum. I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series.
Re:Beringei: A high-performance time series storage engine
by Scaevolus on Saturday February 04, 2017 @06:22AM
(#13566609)
Not quite. Delta of deltas means that periodic timestamps turn into a long run of zeros (since the difference between each point is constant), which is very easy to compress. In the Gorilla paper, 96% of timestamps compress down to zero, which is encoded by a single zero bit.
For series without periodic sampling, delta-of-deltas performs similarly to normal deltas.
Re:Beringei: A high-performance time series storage engine
by rakoo on Saturday February 04, 2017 @07:33AM
(#13566757)
"delta of delta" is only used for timestamps, not for values. In any standard monitoring setup, the acceleration of timestamps will be 0 for a big portion of measurements, and very small for a big portion of the rest, so it makes compression extremely easy and efficient.
Re:Beringei: A high-performance time series storage engine
by throwaway2016a on Saturday February 04, 2017 @11:36AM
(#13567566)
> I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series.
I thought they were talking about healthcare data too when I first read it but seeing as this is Facebook I realized they are talking about server health not human health.
Re:Beringei: A high-performance time series storage engine
by machineman44 on Saturday February 04, 2017 @05:31PM
(#13569496)
Haha, I thought jdonaldson was making a joke :P
Somebody implemented the algorithm in go based on the paper here: https://github.com/dgryski/go-tsz
A short answer that may work for your question: the bits that are set in RAM are xor values relative to previous values. To provide an answer as to what the value is, a series of read|xor operations are performed.