To add to this, the jitter becomes more and more important the larger the scale you are running. The whole calculation (potentially the whole machine) is sitting around waiting for the slowest single task to finish up. Thus you get counterintuitive architectures, like a dedicated core just to handle networking operations. It seems like a bad deal to throw away 6% of your computation, but the alternative is even worse utilization. Highly coupled calculations are a very different beast because they cannot be executed out of sequence.