Ye. I can't stress enough how bad these anti-noise filters are. Just make people mute them self and use push-to-talk if the meeting is too big. Any natural "eehhumhum" before saying stuff goes dead. Surely it adds latency to, since the stream need to process some time before it can decide to start transmitting to everyone.

When I was using self-hosted Ventrilo playing with friends we didn't have this problem. It was like being in the same room. The ping was like 5ms and sound latency not noticeable.