This (plus at least one other here, so it's not really "just you") makes me wonder if you/some people who find this problematic have an issue like central auditory processing disorder.

People with CAPD are often dependent to a high degree on lip reading and video feed may disrupt the connection between words and mouth movements just enough to inject too much noise into the signal. If you don't realize this is what you are doing, you can't readily compensate when it gets disrupted.