I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong. It'll thrash, or crash, or generally have poor performance. We'll be working on improving the failure conditions, but if you put a query to a database that is too big for the server to handle, some sort of failure scenario will be encountered. Just a question of how it's handled.
Also, there's no new query planner in 0.10. We have a bunch of work getting merged in for the query engine for 0.11. But if you throw a huge query at the DB that the server can't handle, you'll still have great sadness.
Which DB ended up solving your big query problems? Maybe we're just a poor fit? Or what kinds of queries should we be working on optimizing?
That means any ad-hoc query can trivially crash your server. That's pretty serious. It's even worse if the ingest is using statsd-influxdb-backend with udp since during the crash you'll lose data.
Do you really think it's ok for a novice user to be able to crash the server from your web gui if they make a minor mistake (e.g something like
SELECT value FROM /series.*/
not realizing how many series their regex actually applies to until it's too late and the has gui stopped responding)?I think it's common in databases that if you throw a massive query at it that the server doesn't have the resources to handle, things will go wrong.
This thinking is worse then wrong. Not only is it wrong - a simple unqualified select won't crash any "common" database - it makes anyone with any "common" database experience doubt your other claims.
This has never ever happened to me in the last 20 years I've used SQL databases. Sure it might take a long time but that's it.
> The current version still has the problem that if you do a massive SELECT then it will fill up memory until the process gets killed. In the future we'll give controls to limit how much memory a query can take.
...
It locks on the query until its killed manually and/or kills it automatically.
It doesn't crash.
Did you folks fix the "oops, I accidentally selected too much and OOMed the machine" crash that was supposed to be resolved by the new query planner in .10? That was my immediate showstopper for InfluxDB for a very high-volume infrastructure, because I didn't feel like proxying InfluxDB just to enforce chunking, and the mere existence of the bug gave me pause. I hit that in 24 hours and shelved the system in 48.
Even pre-1.0, I'm not encouraged when I have to "work around" pretty obvious oversights in reliability, so I put you back on the back burner to give you time to mature and transitioned back to a Kappa-style architecture for my needs. I might still use InfluxDB for low-volume aggregations out of my stream processing, but it's for sure out of the hot path for the foreseeable future.