News for hackers. Stuff that matters.
Re:Lies, Damned Lies, and Averages: Perc50, Perc95 Explained... by kqr on Wednesday March 18, 2020 @06:52AM (#22615256)
> Well, when it comes to performance - you can’t use the average if you don’t know the distribution.

...and if you have the distribution, you no longer need the average!

Latency as experienced by the end user is dominated by the fat fail, for both technical reasons and psychological ones.

Technical ones are probably the most convincing: it is very rare to have users submit single requests and then be done. Especially in the cloudified, service-oriented stacks of today, even single requests lead to a cascade of requests inside the system. Whenever you have tens or hundreds of requests for a single user, it starts becoming very likely that they hit that fat tail at some point in their journey.

Given that latency is dominated by "outliers", looking at anything but p99 and beyond is meaningless.

What's worse is this: since most people at best look at p95 or p99, they tend to optimise for "the common case" at the cost of tail latencies! They introduce insane variance in latencies making benchmarks better, but things actually get worse for real users.

Sorry, this is a pet peeve of mine.

There are no comments on this comment yet.