The longest queue is going to be in front of the tightest bottleneck, but not the central bottleneck critical to your application.
If you find you have a bottleneck halfway through the pipe and you make that bottleneck go away, your true real bottleneck down in the wall still remains there, and all your optimizations done above that one will end up just opening the floodgates wider.
Once you've protected that kind of more at-risk core, then it's easier and safer to measure everything that sits above it in the stack, and it puts an upper bound on how much you need to optimize (you stop when you hit your rate limits on the core component).
Re:Queues Don't Fix Overload
by jacques_chester on Thursday November 20, 2014 @04:36PM
(#8638309)
Dumb question: how does one decide which is the "true" bottleneck? There's always a bottleneck, surely.
I'm reminded of Weinberg's quip that solving your most pressing problem just promotes #2 into first place.
Is this a case for using queueing theory to model your system's behaviour?
Even if you don't go down that path (and there are tools for it[0]), using queues to connect components helps find bottlenecks in one very simple way: you can look for where the queues are growing. As a rule of thumb, the longest queue is going to be in front of the bottleneck.
[0] http://www.perfdynamics.com/Tools/PDQ.html