Try to replicate the freedom (no-schema, cardinality, adding a new node to cluster) of Elasticsearch with InfluxDB or others and you will hit cardinality problems real fast.
Open to hearing other opinions though.
The result is a dashboard that takes ages to load just to show a trend in values.
I think it's using the wrong tool for the job, but maybe it's just me.
Immutable time series data like logs and metrics are a great fit for Elasticsearch due to the way Lucene stores data. Documents in Lucene are immutable so an update in Elasticsearch is creates a new document and places a tombstone marker on the old one. Immutable data means you don't have to tolerate those inefficiencies.
Dashboards don't load the entire dataset by default. I can't remember what the exact default time range is but I think it's ~15 minutes or so. They're fairly quick to render in Kibana.
Elasticsearch is a great tool for observability data (logs, metrics, and APM data). Elastic's tooling makes a lot of this really easy in most cases.
One of the main strengths of Elasticsearch is that you can use it for searching and aggregating in a single query. But you need to ensure you are searching on fields that are indexed for search and sorting/aggregating on fields that are indexed for that.
any idea why? Seems like an odd approach to me.
It’s generally considered too expensive to derive metrics from logs in this way, but it’d be a killer debugging experience.
It seems to commonly be perceived as a panacea of databases.
If you're not doing free-text search, and your data will fit in memory in a big relational database VM for the foreseeable future, why Elasticsearch?