YMMV. I can relate. When I'm stressed for various non-noise related reasons, I am more easily irritated by noises, and I usually, initially, assume it's the noise.

On the opposite note, I can perform extremely well in noisey environments if I am enjoying what I am doing - like I am able to just tune it out.

Circumstance and other stressors may be a main factor, or it could just be the noise. I imagine it really depends on the person.