News for hackers. Stuff that matters.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by ddorian43 on Thursday February 04, 2016 @03:10PM (#11036797)
They did the mongodb approach. Build a shitty db and then build a normal one and claim 100X faster.

And then add synchronized disk commits (like postgresql has always done) and performance goes puff (meaning lower than pg) (example: elasticsearch, mongodb).

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by pauldix on Thursday February 04, 2016 @03:14PM (#11036822)
fsyncs happen on every write. They're durable. This release may be 100x faster than our previous release, but it's also a competitor with other solutions.

We haven't run any comparisons vs. other solutions, but we will do that soon and I expect that we'll be competitive and better in some cases.

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by CyrusL on Thursday February 04, 2016 @03:16PM (#11036844)
Very tasteful way to handle the parent comment :)

Does Todd Persen talk about durability testing in the video or is more about performance benchmarking? I haven't watched yet.

https://vimeo.com/153802549

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by pauldix on Thursday February 04, 2016 @03:20PM (#11036871)
Thanks :). He talks about doing kill nines on the process while doing a bunch of writes, trying to corrupt the DB, and perf testing. In addition to a little bit of talk about what came before and some lessons learned.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by SEJeff on Friday February 05, 2016 @12:12AM (#11039630)
Have you had a chance to throw Influx through the paces of Kyle Kingsbury's Jepsen distributed data store tester?
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by pauldix on Friday February 05, 2016 @12:22AM (#11039674)
Not yet, but it's on our TODO list. However, we have to finish the clustering implementation before we bother going through that testing. At this point we're telling people it's experimental and not meant for production use.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by fapjacks on Thursday February 04, 2016 @03:38PM (#11037022)
This is not the first time I have seen an anti-InfluxDB post that also subtly pushes PostgreSQL in its place. Now I totally agree and I am not happy with InfluxDB's performance, and maybe I'm just being paranoid, but it's interesting I've seen this a handful of times.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by ddorian43 on Thursday February 04, 2016 @03:49PM (#11037104)
Nothing against influxdb. But if you have 100X, you probably were doing something shitty before.

I use postgresql as comparison because antirez did in http://oldblog.antirez.com/post/redis-persistence-demystifie... (and ~none else has done something similar).

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by pauldix on Thursday February 04, 2016 @03:47PM (#11037091)
Have a look at this release. The write performance is massively better than it was. We did tests writing up to 100B points against a single server.

The next release will focus on improving query performance, but this one still works for many cases.

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by officialchicken on Thursday February 04, 2016 @04:25PM (#11037362)
"Massively better" doesn't mean deployable to production either. Especially not if you've only tested with a single point of failure with a single sized payload. But I'm willing to give it a third (and final) trial:

1) Does "DELETE * FROM foo" still cause the system to lockup, freeze, and require a restart to free memory? Or are there other conditions/queries that cause the system to become unstable?

2) There's no README/CHANGELOG/dependencies info on the download page. Which is the preferred version of Go to install on my servers for Influxdb - 1.4 or 1.5?

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by jtblin on Friday February 05, 2016 @12:09AM (#11039617)
I'd guess go version doesn't matter, you should install the binary which is statically linked, no?
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by cbsmith on Thursday February 04, 2016 @05:22PM (#11037774)
Massive performance improvements like that tend to come down to overall changes in the data structure and/or algorithm. This is like, "oh yeah, maybe we should do LSM?" or "maybe compressed bitmaps?" kind of stuff. What what the magic?
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by CyrusL on Thursday February 04, 2016 @05:27PM (#11037809)
New storage engine.

Introduction from last October: https://influxdata.com/blog/new-storage-engine-time-structur...

Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by cbsmith on Thursday February 04, 2016 @09:42PM (#11039052)
Oh right, I remember when that blog post came out. It was amusing because I'd built an in-house metric store much like InfluxDB (using a LevelDB engine), and reached much the same inflection point and decision. I so should have open sourced that so you poor saps didn't have to experience my pain.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by dbettin on Thursday February 04, 2016 @03:54PM (#11037139)
What is the query performance like today? And, what are your goals for query performance after the next release?
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by flowless on Thursday February 04, 2016 @03:47PM (#11037089)
I would like to see actual comparison of both. I'm going pg way myself but due to millisecond precision of timestamps required for handling scientific data.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by pauldix on Thursday February 04, 2016 @03:50PM (#11037113)
InfluxDB supports timestamps down to the nanosecond. We recommend that you only use the precision you need, since you'll get better compression that way.
Re:InfluxDB v0.10 GA with hundreds of thousands writes/sec, ... by flowless on Thursday February 04, 2016 @04:13PM (#11037262)
Good to know, thanks!