It could apply to me but I run evals every time someone at work complains about this and it’s never shown any more or less variance than regular usage. I don’t have statistical relevance with my evals but I do have a decent history with them.

People seem to underestimate how high normal variance is with these tools.