I knew someone would say something like that, but I've never seen that sort of thing done personally and I really doubt lack of statistical knowledge is even close to the biggest obstacle to writing faster software. For micro-optimizations so subtle you need fancy techniques to even tell they work, non-quantitative factors (code impact, will it enable other optimizations, etc.) are more likely to be decisive. Techniques to reduce noise are either non-statistical (warming up caches) or unsophisticated (average many trials, best of three).