News for hackers. Stuff that matters.
Re:Grafana Mimir – Horizontally scalable long-term storage f... by eatonphil on Wednesday March 30, 2022 @09:27AM (#30854914)
There isn't a link to the project on the page (that I could find) so it almost looked like it's not open source. But here it is: https://github.com/grafana/mimir.
Re:Grafana Mimir – Horizontally scalable long-term storage f... by notamy on Wednesday March 30, 2022 @09:33AM (#30854994)
You have to find the "Download" button and click it, it's very non-obvious :< The entire page seems to be designed to funnel you into signing up for their paid service, which makes sense, but still doesn't feel great...
Re:Grafana Mimir – Horizontally scalable long-term storage f... by candiddevmike on Wednesday March 30, 2022 @09:41AM (#30855067)
Recently switched from their cloud service back to on-premise. The cloud version wasn't being updated and the entire setup experience left a lot to be desired with how you connect their on-premise grafana agent, especially if you aren't using their easy button deployment stuff. Also, billing for metrics is insane, as on any given day my metric load may vary between 5-7k or more. This caused some operational overhead as I was constantly tweaking scrapers to reduce useless metrics.

For $50/mo, you can self host everything easier, cheaper and with more control IMO.

Re:Grafana Mimir – Horizontally scalable long-term storage f... by maccard on Wednesday March 30, 2022 @09:54AM (#30855272)
> For $50/mo, you can self host everything easier, cheaper and with more control IMO.

Can you give an example as to how you could self host a grafana stack for $50/month? On AWS that buys you 4 cores, 8GB memory and 0 storage, and it's certainly not easier than clicking one button on the grafana website.

Re:Grafana Mimir – Horizontally scalable long-term storage f... by m1keil on Wednesday March 30, 2022 @10:31AM (#30855707)
We are running Grafana and Prometheus on a single t3.xlarge instance with 150GB gp3 EBS.

Excluding traffic, it costs ~ $100 USD per month.

We are doing 10 second scrapes and currently have roughly 141k active time series. In Grafana Cloud it would cost...

15000 metrics for free. 126000/1000 * $8 = $882

Now here's the real kicker.. the pricing Grafana puts on their website are assuming 60 second scrape interval (1 data point/minute or DPM). If you are doing 6 DPM, that's $8 * 6 per 1000 time series!

So final bill.. drum rolls

126000/1000 * $8 * 6 = $6048

Yes. That's a 60x.

Now, sure, we don't get the scale, the backups, the SLA.. but we can live without it. And when Prometheus will start acting slowly, we will just bump it to t3.2xl, or spend some time and filter out some of the noisy metrics we might have around.

Btw, if you try to find any information about what is a "time series" or a "metric" on the Grafana's pricing page, good luck.

https://grafana.com/docs/grafana-cloud/metrics-control-usage...

Re:Grafana Mimir – Horizontally scalable long-term storage f... by maccard on Wednesday March 30, 2022 @10:36AM (#30855787)
> Excluding traffic, it costs ~ $100 USD per month

I don't doubt that that's affordable, or cost competitive to AWS, but thats' about as cheap as you can do it, _and_ that's not including traffic. It's pretty much impossible to half that bill.

Re:Grafana Mimir – Horizontally scalable long-term storage f... by m1keil on Wednesday March 30, 2022 @10:59AM (#30856087)
I excluded the traffic because the price is basically 0. This is internal traffic and a bunch of HTTP requests. It doesn't cost us $3000 a month.
Re:Grafana Mimir – Horizontally scalable long-term storage f... by alexjplant on Wednesday March 30, 2022 @10:10AM (#30855455)
There are Helm charts available for all Grafana products so if you already run a Kubernetes cluster and have spare capacity you can just throw it up there. Loki supports shipping logs to GCS/S3 natively and Prometheus can use Cortex (also available as a Helm chart) to do the same. Once you throw Grafana behind SSO and implement a backup cronjob you're done until you reach scale and have to start deploying/scaling individual components separately.

I implemented most of the above using Terraform on a managed DigitalOcean cluster on a Saturday a few months back; it wasn't super-hard. Alternatively you could rent a few VPSes someplace and use k3s or similar to get an unmanaged cluster.

Re:Grafana Mimir – Horizontally scalable long-term storage f... by westurner on Wednesday March 30, 2022 @12:34PM (#30857512)
Suggestions for organizing a Helm + Terraform [+ k3s/k3d/MicroShift] provisioning and monitoring git repo with CI for job accounting? (without Ansible & AWX, which I'd create a role with for this too)

- [ ] ENH,BLD: A cookiecutter for this would be cool

Re:Grafana Mimir – Horizontally scalable long-term storage f... by krnlpnc on Wednesday March 30, 2022 @10:40AM (#30855839)
> $50/month? On AWS that buys you 4 cores, 8GB memory and 0 storage

Self-hosting on AWS is kind of counterproductive. Look into "cloud" metal servers and the money will go much further.

Re:Grafana Mimir – Horizontally scalable long-term storage f... by jrwr on Wednesday March 30, 2022 @10:09AM (#30855445)
Two low end Hetzner/OVH Boxes for redundancy should do the trick
Re:Grafana Mimir – Horizontally scalable long-term storage f... by jjeaff on Wednesday March 30, 2022 @08:39PM (#30862567)
That's why AWS charges so much for outgoing traffic.
Re:Grafana Mimir – Horizontally scalable long-term storage f... by dewey on Wednesday March 30, 2022 @09:38AM (#30855042)
The first CTA button on the page "Tutorial" links to a tutorial where the first step is to run the project with Docker. Doesn't really feel like an overly forced funnel to their paid service.
Re:Grafana Mimir – Horizontally scalable long-term storage f... by mdaniel on Wednesday March 30, 2022 @04:32PM (#30860425)
Still AGPL, which I guess makes sense given the rest of their stack is too: https://github.com/grafana/mimir/blob/mimir-2.0.0/LICENSE