News for hackers. Stuff that matters.
Re:Lies, Damned Lies, and Averages: Perc50, Perc95 Explained... by alexhutcheson on Thursday March 19, 2020 @10:26AM (#22627760)
My normal approach is to measure and set up separate alerts for "error rate" and possibly "timeout rate". You definitely want to know about those, but "mean latency" mixes those metrics with the latency metrics for successful requests, which makes it less sensitive to changes in either one.

In general I agree that averages aren't always bad. One additional advantage is that it's often possible to generate robust confidence intervals for averages, but it's often not valid to generate CIs for medians/percentiles without introducing other probably flawed assumptions.

Re:Lies, Damned Lies, and Averages: Perc50, Perc95 Explained... by contravariant on Thursday March 19, 2020 @10:37AM (#22627866)
Error rate and timeout rate are good examples of an average that measures the thing you're actually interested in.

The whole point is that trying to average the latency and only then try to figure out what it means is backwards.