Re:Stabilizer: Statistically sound performance evaluation [pdf]
by titzer on Tuesday February 26, 2019 @04:28AM
(#19253030)
> why do those things have a (statistically significant?) impact in the first place?
In a word, caches. Not just the instruction / data cache, but also page faults and micro-architectural features like micro-op caches, instruction TLB entries, loop stream buffers, cache-line alignment, and aliasing in the branch predictor tables (which can also be thought of as caches).
Re:Stabilizer: Statistically sound performance evaluation [pdf]
by peterwaller on Wednesday February 27, 2019 @08:16AM
(#19263008)
I suppose I was musing more along the lines of "why isn't this a solved problem". Clearly, it isn't an easy one or compilers would already take this into account and then the statistical variance would be reduced.
Re:Stabilizer: Statistically sound performance evaluation [pdf]
by gnufx on Tuesday February 26, 2019 @12:51PM
(#19256606)
Yes, it's important to measure and control the effects of things like topology/affinity, which can have factors of several effects on HPC program efficiency. You shouldn't suffer much variance on single-node compute, but it's different when you're subject to effects on/of the network fabric and filesystem.
It could be that the problem arises when trying to measure very small speed increases (small relative differences => noise matters more). But in that case the fact that such a small speed increase is wiped out by random layout effects surely means that time would be better invested in finding a more performant layout?