Re:Lies, Damned Lies, and Averages: Perc50, Perc95 Explained...
by djspoons on Thursday March 19, 2020 @11:20AM
(#22628293)
Well, yeah, there's not much difference between a histogram and hundreds-of-percentiles. But there's a huge difference between a histogram and just a single percentile (whether it's p50, p95, p99, or whatever).
If you just want to know _if_ something changed then maybe one percentile is ok (though actually: not great). But if you are trying to figure out _what_ happened (or is happening!), a histogram is really important. Most of the time there are discrete behaviors/factors that are driving performance: timeouts, cache hits/misses, one overloaded host, a canary. The shape of the histogram will help you see those in a way that a single percentile can't.
Re:Lies, Damned Lies, and Averages: Perc50, Perc95 Explained...
by djspoons on Thursday March 19, 2020 @11:45AM
(#22628536)
Actually one of my coworkers gave a talk at Facebook Performance summit on this:
https://www.youtube.com/watch?v=EG7Zhd6gLiw
(Disclaimer: brief product pitch in the first minute as part of the speaker intro)
In fact, the only big difference between histograms and percentiles are that they're sort of the same thing on a different axis.