News for hackers. Stuff that matters.
Re:InfluxDB v0.9.0 released with developer and production su... by SEJeff on Thursday June 11, 2015 @12:47PM (#9700784)
Are there any plans to run @aphyr's jepsen CAP testing system on influx? It doesn't give me a ton of confidence that you're redoing your clustering pretty much from scratch a 3rd time in a pre-1.0 piece of software.

It would also be excellent if you could post more on the not really CA and not really AP aspect of your new clustering design, and why no one has really done that before (from what I can tell). Why that is the best path forward.

Thanks for releasing such awesome OSS software.

Re:InfluxDB v0.9.0 released with developer and production su... by snowwindwaves on Thursday June 11, 2015 @01:15PM (#9700987)
I never nail it the first time around. The fact that they have explored the problem three times and revised their solution in pre 1.0 piece of software actually comforts me. pre 1.0 is a great time to make big revisions like this. It may have been close to good enough the first time, but the third iteration will undoubtedly be better. From @aphyr's blog it seems that other projects have issues with clustering, so the fact that influx's clustering design is unique isn't necessarily a detracting factor. It isn't not re-inventing the wheel if there is no tried and true solution they could be using.
Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @12:50PM (#9700796)
We will be doing that testing in the future. I've already talked to Kyle about our design. We'll iterate on his and other smart people's feedback. That's one of the reasons the clustering alpha right now :)
Re:InfluxDB v0.9.0 released with developer and production su... by SEJeff on Thursday June 11, 2015 @01:52PM (#9701209)
This seems reasonable, thanks. Keep the blog posts coming, they're great!
Re:InfluxDB v0.9.0 released with developer and production su... by sciurus on Thursday June 11, 2015 @12:59PM (#9700871)
For people curious where the clustering support stands:

"From the InfluxDB clustering design doc we’ve implemented the write path to replicate data to other nodes. We’ve implemented the per write request consistency levels and hinted handoff, ensuring that server reboots and short outages are quickly recovered from and consistency is restored.

What’s missing are anti-entropy, distributed queries, joining new servers to a cluster, and tools for replacing servers in a cluster. We’ll start development on some of these features in the 0.9.1 release cycle and more in the 0.9.2 cycle. They will be merged into releases once they’re stable and tested."

Re:InfluxDB v0.9.0 released with developer and production su... by jbrantly on Thursday June 11, 2015 @12:50PM (#9700797)
Super excited about this. Congratulations on the release!

I've seen Prometheus bash a little on InfluxDB regarding storage requirements [1]. Is this something that has been addressed? Is it even something worth worrying about?

1. http://prometheus.io/docs/introduction/comparison/#data-mode...

Re:InfluxDB v0.9.0 released with developer and production su... by jrv on Thursday June 11, 2015 @01:10PM (#9700948)
Last I checked that still held true. Here's a more up-to-date version of those benchmarks: https://groups.google.com/forum/#!searchin/influxdb/promethe...

For Prometheus's long-term storage I'd basically like to have something like InfluxDB (Go, simple to operate, distributed storage), but better suited towards purely numeric time series rather than arbitrary log records. Unfortunately as it stands, InfluxDB takes roughly 14x more disk space for exactly the same data (without replication) for a typical Prometheus use case. I hope that improves in the future, but InfluxDB's use case is also different: they enable you to store arbitrary key=value metadata with every single record, rather than defining a time series once and then only appending numeric samples. So the comparison is not really fair, it's just maybe not the right tool for Prometheus-style monitoring.

Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @12:54PM (#9700833)
We will be starting work on a new storage engine or an optimized version of what we have now that will add compression. Tentatively, that will start in the 0.9.3 release cycle.

First rule of software: make it work, then make it efficient

Re:InfluxDB v0.9.0 released with developer and production su... by misframer on Thursday June 11, 2015 @12:55PM (#9700843)
I hope you guys collaborate with the RocksDB team. Column families [0] seem like a great fit for time series.

[0] https://github.com/facebook/rocksdb/wiki/Column-Families

Re:InfluxDB v0.9.0 released with developer and production su... by avolcano on Thursday June 11, 2015 @12:27PM (#9700634)
Blog post with more useful information: http://influxdb.com/blog/2015/06/11/InfluxDB-v0_9_0-released...

The lack of an upgrade path from 0.8.x makes me feel pretty stupid for trying out that version. Then again, they don't have upgraded client libraries for Python either, so we wouldn't be ready to upgrade anyways...

Re:InfluxDB v0.9.0 released with developer and production su... by sctb on Thursday June 11, 2015 @12:29PM (#9700642)
Re:InfluxDB v0.9.0 released with developer and production su... by vruiz on Thursday June 11, 2015 @12:34PM (#9700673)
Finally! congrats @pauldix & co. Go checking that influx 0.9 wasn't out yet was the only thing that made me feel better about missing deadlines at work, looks like I'll have to find something else :)
Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @04:45PM (#9702292)
Sorry to ruin a perfectly good excuse :)

Lies, damn lies, and delivery estimates

Re:InfluxDB v0.9.0 released with developer and production su... by dudul on Thursday June 11, 2015 @03:53PM (#9701919)
How can you release a product with such an outdated doc? Seriously, I literally copy-pasted the example of data insertion from the github README and I get 'Time Serie Name is required.'

The Getting started mentions the line protocol, and the link to 'learn more about it' talks about the POST body being JSON (a malformed JSON cause I get the same error in the UI when trying to insert it). Which one is it?

Re:InfluxDB v0.9.0 released with developer and production su... by koppor on Tuesday June 16, 2015 @01:18PM (#9727340)
The libraries also weren't updated :) -> see https://github.com/influxdb/influxdb-python/issues/193

The issue behind is that "name" has been renamed to "meassurement" - https://github.com/influxdb/influxdb/issues/2564. They implemented that change on 2015-05-21 in rc31

Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @04:15PM (#9702079)
We'll update the readme. Overlooked in the push to release. The docs on InfluxDB.com should be accurate
Re:InfluxDB v0.9.0 released with developer and production su... by dozzie on Thursday June 11, 2015 @12:48PM (#9700789)
I wonder when they are planning to replace their shitty DEB and RPM packages with something built properly and with proper post-install and init scripts. Currently they're using ridiculous interactive script that clamps together fetching code, compiling it, packaging artifacts (in fact, it contains some artifacts (sic!)), and shipping packages to Amazon.
Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @12:51PM (#9700809)
Have a recommendation for what we should be doing? More than happy to update how we package to make things easier or more idiomatic
Re:InfluxDB v0.9.0 released with developer and production su... by dozzie on Thursday June 11, 2015 @01:44PM (#9701157)
1. Split all the above-mentioned steps to separate scripts or makefile targets. It's ridiculous to mix them; how are people supposed to build InfluxDB inside networks that don't have access to internet?

2. Learn how to build RPMs and DEBs the proper way (Debian source package, SRPM) instead of cutting corners with FPM. It fails to include dependencies, and full description, for instance, or to mark files as configuration.

And yes, your binaries do have dependencies.

Also, your DEB metadata is broken, according to lintian.

3. Learn where to put which files in Linux. LSB wasn't agreed on for nothing.

4. Rewrite your post-install script to something that fits your target distributions' conventions. This most probably means two scripts, for Red Hat and for Debian. For instance, under Debian your packages create service user that doesn't fit the rest of the system (UID above 500, home under /home, usable shell).

Re:InfluxDB v0.9.0 released with developer and production su... by justinsb on Thursday June 11, 2015 @02:35PM (#9701428)
I've wanted to create (deb) packages in the past, but found a large number of conflicting instructions on the "right way". Is there an approach / howto-doc you would recommend?
Re:InfluxDB v0.9.0 released with developer and production su... by jpgvm on Thursday June 11, 2015 @02:47PM (#9701495)
The right way is to provide a debian directory with the appropriate packaging metadata.

Use git-buildpackage and associated tools (git-dch etc) to build the packages natively (or allow others to build for their Debian based distros).

Re:InfluxDB v0.9.0 released with developer and production su... by justinsb on Thursday June 11, 2015 @04:51PM (#9702345)
Thanks, but it's still not obvious to me what to do. Looking at this: https://wiki.debian.org/PackagingWithGit, git-buildpackage documentation seems to suggest either creating a separate branch, or packaging tarballs, both of which seem different to what you're suggesting where I just create a debian directory.

Do you have a different document or an example repo with the debian/ directory? This is why I've resorted to FPM in the past - the deb tools seem so simple, and I'm sure they are simple for people that have done it before, but the documentation for FPM is much more direct and thus people (such as myself) use it, even though we know it's not "right".

Re:InfluxDB v0.9.0 released with developer and production su... by dozzie on Thursday June 11, 2015 @05:21PM (#9702534)
Just like the large number of conflicting instructions for building iptables firewall, none of which got it how to write a set of rules and instead focuses on how to write a set of commands. One just needs to apply common sense instead of blindly follow somebody's recipe.

But why not just use the introduction from the official documentation?

https://wiki.debian.org/IntroDebianPackaging

Re:InfluxDB v0.9.0 released with developer and production su... by justinsb on Thursday June 11, 2015 @05:38PM (#9702622)
I find that introduction is intended for people packaging software that other people have written, where upstream creates a tarball.

What FPM nails is the use-case where a developer wants to package their own software. Obviously I could take on the role of upstream, publish .tar.gz files, and then take on the role of packager and consume those, but I think you can understand at that point why FPM looks so attractive!

You obviously know this area well: a link to software you have written and packaged would be greatly appreciated.

Re:InfluxDB v0.9.0 released with developer and production su... by XorNot on Thursday June 11, 2015 @02:00PM (#9701269)
It'd also be nice to have official systemd service files. With Debian 8 out there its basically necessary now.
Re:InfluxDB v0.9.0 released with developer and production su... by veeti on Thursday June 11, 2015 @01:03PM (#9700896)
Has anyone been adventurous enough to use the 0.9 release candidates or previous versions to collect and analyze application & server metrics with statsd, Grafana, etc.? How has it worked for you? Any recommendations?

I've been waiting for 0.9 to hit, so I guess I'll find out myself soon enough.

Re:InfluxDB v0.9.0 released with developer and production su... by erewh0n on Thursday June 11, 2015 @01:36PM (#9701106)
I've been running infxludb 0.9 + heka 0.9.x + grafana 2.x stack in a production environment for over a month. Mostly successful, although I find the grafana tooling for building queries to be a bit weak. I think there are improvements coming soon to address this, though. Overall, influx performance seems to be pretty good although we aren't ingesting enough data to say anything really concrete. We also make use of continuous queries to aggregate our time series, which of course makes large-interval queries more efficient.
Re:InfluxDB v0.9.0 released with developer and production su... by tehno on Friday June 12, 2015 @09:05AM (#9705580)
We've been running Influxdb 0.9 (pre-rc32), Heka 0.9 and Grafana 2 as well. It's been going OK, though we really needed to add stats collecting and batch write in Heka layer to not spike the CPU on writes to Influxdb.

The data write API changes so late in the 0.9 release candidates phase was an unexpected surprise. Also, needing to throw away all data written using pre-rc32 versions sucks. No way to migrate db versions? Boo!

Re:InfluxDB v0.9.0 released with developer and production su... by switch007 on Thursday June 11, 2015 @01:13PM (#9700970)
My server ran out of disk and now I can't 'list series' in InfluxDB and my graphs are empty. Need to investigate...

(0.8.8 + statsd + grafana)

Re:InfluxDB v0.9.0 released with developer and production su... by Mahn on Thursday June 11, 2015 @01:40PM (#9701133)
Minor typo in http://influxdb.com/support/

"Purcahses" for InfluxDB development support provide...

Re:InfluxDB v0.9.0 released with developer and production su... by detaro on Thursday June 11, 2015 @04:03PM (#9701990)
Also, their support options don't seem to have prices? Why publish a list of different tiers if you have to ask for a quote anyways?
Re:InfluxDB v0.9.0 released with developer and production su... by deltamidway on Thursday June 11, 2015 @12:43PM (#9700756)
Go Paul and Todd! Go!
Re:InfluxDB v0.9.0 released with developer and production su... by pspeter3 on Thursday June 11, 2015 @07:04PM (#9703017)
Homebrew still has version 0.88. Will 0.9 be coming soon?
Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @09:20PM (#9703476)
yep, should be getting updated in the next day or so. We're on it ;)
Re:InfluxDB v0.9.0 released with developer and production su... by XorNot on Thursday June 11, 2015 @12:26PM (#9700630)
The download link is currently throwing access denied :(
Re:InfluxDB v0.9.0 released with developer and production su... by pauldix on Thursday June 11, 2015 @12:40PM (#9700735)
looking at it, should be fixed in a few minutes
Re:InfluxDB v0.9.0 released with developer and production su... by SuchHappyDude on Thursday June 11, 2015 @02:13PM (#9701312)
wouldn't pay for a developer plan when the docs are so out of date. Update the docs, and I'll see what I really need paid support for.