Re:Graph everything with Graphite
by KaiserPro on Friday December 12, 2014 @09:54AM
(#8740277)
Graphite is awesome:
We graph lots and lots of things. we have two datastore servers, and each of them has a static write load of about 60 megs a second (bear in mind that each update is less than 100bytes) we have many thousands of updates a second.
But why is it awesome? because it almost eliminates the need for log shipping. 90% of the time we can diagnose most problems with just graphs. Something is running slow? well we can see which server it is by looking at load, response time and queue size.
Because we are not doing silly things like parsing logs to gain metrics, we don't need a hadoop system. We plumb metrics collection directly into Cgroups (the primitive that docker uses) so we can get per process metrics (disk, memory, cpu etc)
The only time we need logs are when we are really stuck, or diagnosing a specific issue like "why" it went wrong, not what has gone wrong.
But why is it awesome? because it almost eliminates the need for log shipping. 90% of the time we can diagnose most problems with just graphs. Something is running slow? well we can see which server it is by looking at load, response time and queue size.
Because we are not doing silly things like parsing logs to gain metrics, we don't need a hadoop system. We plumb metrics collection directly into Cgroups (the primitive that docker uses) so we can get per process metrics (disk, memory, cpu etc)
The only time we need logs are when we are really stuck, or diagnosing a specific issue like "why" it went wrong, not what has gone wrong.