Consider that as audio quality gets worse and worse eventually becoming indecipherable (from latency, dropouts, distortion...), it takes increasing effort to understand what is being said. The curve is non-linear, but the direction of the correlation is clear.

There is no question that auditory fatigue is real. The question is only to what extent poor audio is a contributing factor to "Zoom Fatigue".