News for hackers. Stuff that matters.
Re:Cortex: a multi-tenant, horizontally scalable Prometheus-... by netingle on Wednesday December 19, 2018 @06:19PM (#18720908)
Nice! I agree Cortex's operational overhead can be quite high. Good news! We are going to fix that. We've got PRs open to add local storage (based on BoltDB) to remove the dependancy on DynamoDB/Bigtable, and I've almost finished a single binary, non-microservices version of Cortex.

If you want a sneak peak, checkout grafana/loki; its based on Cortex and has all theses features - buts it for logs, not metrics.

Re:Cortex: a multi-tenant, horizontally scalable Prometheus-... by grumpydba on Thursday December 20, 2018 @02:29AM (#18723100)
Hi,

It is my understanding that Prometheus's storage engine is LSM based, and uses a compression scheme inspired by gorilla.

Boltdb is basically a btree which causes more write amplification.

Maybe you should try badgerdb.

Re:Cortex: a multi-tenant, horizontally scalable Prometheus-... by bboreham on Thursday December 20, 2018 @03:39AM (#18723352)
Cortex also uses gorilla-inspired compression, using the Prometheus source code.

After compressing timeseries they are stored and indexed using a database-like thing, typically DynamoDB or Bigtable, but soon BoltDB will be an option.

(I am a Cortex maintainer)