News for hackers. Stuff that matters.
Re:Time series database Graphite seems to be falling into di... by cbsmith on Sunday November 08, 2015 @11:35PM (#10531414)
I highly recommend looking at Keogh's work, particularly the iSAX and iSAX2 stuff: http://www.cs.ucr.edu/~eamonn/SAX.htm

While it is used for machine learning, it actually makes a lot of sense to follow a similar approach for more general time series applications.

Re:Time series database Graphite seems to be falling into di... by gtrubetskoy on Sunday November 08, 2015 @11:53PM (#10531459)
I'm familiar with SAX, it's remarkably powerful at discerning patterns and identifying similarity between series, good stuff, thanks for the link.
Re:Time series database Graphite seems to be falling into di... by chaotic-good on Monday November 09, 2015 @03:48AM (#10531953)
SAX requires different type of storage. You can store SAXified time series in ElasticSearch, or Solr but time-series database doesn't fit for this. Time-series databases should be able to generate SAX representation.
Re:Time series database Graphite seems to be falling into di... by cbsmith on Monday November 09, 2015 @11:04AM (#10533773)
> You can store SAXified time series in ElasticSearch, or Solr but time-series database doesn't fit for this.

That's making some presumptions about the time-series database use case. SAX is convenient for storing and retrieving the data as well as identifying trends or recurring behaviour. What more do you need?

Re:Time series database Graphite seems to be falling into di... by chaotic-good on Monday November 09, 2015 @12:38PM (#10534439)
You can't retrieve original time-series data from SAX storage because of normalization. To query time-series data by content (approximate 1-NN, motif discovery, etc) you need inverted index.
Re:Time series database Graphite seems to be falling into di... by cbsmith on Tuesday November 10, 2015 @11:56AM (#10540473)
The normalized data is the index. It can still be pointing to the raw data. The nice thing is that the index organizes the data in a way that makes it easily (lossless) compressible.
Re:Time series database Graphite seems to be falling into di... by chaotic-good on Wednesday November 11, 2015 @05:35AM (#10545893)
> It can still be pointing to the raw data.

Yep. Each SAX word should be mapped to the list of seriesid:timestamp pairs. This list is often referred as postings list in information retrieval. The resulting data-structure is an inverted index. SAX and iSAX papers describes inverted index variant (really bad one) based on folder structure but one can use convenient IR tools for this.

Re:Time series database Graphite seems to be falling into di... by cbsmith on Wednesday November 11, 2015 @10:09PM (#10551062)
iSAX2 yields a much better inverted index style model.

The thing is, particularly with time series data, a lot of times it is sufficient to at least start with the summary data in the index.