Re:InfluxDB – Open-source distributed time-series, events, a...
by jwatte on Tuesday November 05, 2013 @02:32PM
(#6677537)
You didn't look at istatd?
It does 150000 counters at three different retention intervals every ten seconds, with average, total, standard deviation, mom and max for all metrics.
We looked at other options (including open tsdb and graphite) before building this.
Https://github.com/imvu-open/istatd
Re:InfluxDB – Open-source distributed time-series, events, a...
by JulianMorrison on Tuesday November 05, 2013 @07:42PM
(#6679701)
https://github.com/imvu-open/istatd/wiki is my favouite. C++, compile to a binary and then it can send updates in a tree of agents to a master node, and maintain a replica of the master, and it does graphs too, everything you need in one.
Re:InfluxDB – Open-source distributed time-series, events, a...
by philjackson on Tuesday November 05, 2013 @08:31PM
(#6679921)
Rolling your own:
http://blog.apiaxle.com/post/storing-near-realtime-stats-in-...
Re:InfluxDB – Open-source distributed time-series, events, a...
by e12e on Wednesday November 06, 2013 @12:38AM
(#6680926)
I should probably look at the code, but reading the post it isn't entirely clear to me: are you updating all three (second, minute, hour) keys on every insert? And are you just using the "last value" as the value for the key (so the value for minute becomes the last value of second that falls in that interval?
I need to read up on rrdtool as well, but I wonder if it would make much difference (good or bad) to store the mean or other average as the "higher up value" (ie: the average of the past 60 seconds as the minute value) ?
Re:InfluxDB – Open-source distributed time-series, events, a...
by vosper on Tuesday November 05, 2013 @01:31PM
(#6677005)
Thanks for sharing all these links! I'm always trying to find new and interesting DB technologies, especially time-series.
Can you share anything of your experiences with kairosdb so far - what's the use case and how has it performed?
Re:InfluxDB – Open-source distributed time-series, events, a...
by hoopism on Tuesday November 05, 2013 @02:06PM
(#6677306)
Thanks! We also went through a similar assessment and came away with kairosdb being the favorable option. Have not started any integration so early thoughts would be helpful. We def favored cassandra over HBASE and the little work we did with opentsdb went well.
Re:InfluxDB – Open-source distributed time-series, events, a...
by cnlwsu on Tuesday November 05, 2013 @03:45PM
(#6678141)
Actually, as a person who used most of these and built things similarly this project seems very interesting and useful and tackles a lot of the annoyances of those above. Kinda a combination of otsdb and esper but without a big requirement of setting up other technologies (I don't have to become or hire a HBase expert to maintain it). Having a bunch of online algorithms executed against the data is exciting too from a perspective of future improvements alsom I would love to see anomaly detection built in.
I think there is a hackernews rule someplace that a more interesting tech alternative shows up right when you decided to go with something else.
For reference here is the list I created when researching these:
http://opentsdb.net/overview.html Built on HBASE
http://www.gocircuit.org/vena.html Built on go + go'circult uses google's leveldb
https://code.google.com/p/kairosdb/ A rewrite of opentsdb which can use Cassandra
http://blueflood.io/ Built by Rackspace, decent but still seems a bit immature
http://graphite.wikidot.com/ Obligatory Graphite reference (uses whisper, new backend called 'ceres' is being developed)
https://github.com/agoragames/kairos (yet another 'kairos', alternative backends for graphite - SQL, redis, or mongo)
Riak seems to be popular with SASS metric providers (hosted graphite, boundary). There isn’t any code but there are a couple of talks that explain how and why they went with Riak:
http://basho.com/hosted-graphite-uses-riak-to-store-all-cust... http://boundary.com/blog/tag/tech-talks/
http://vimeo.com/42902962