That's not really an issue imo, the downside is mostly just getting part of the headset gross (you don't want to use it right after someone else has sweated up a storm in there, believe me).