I suppose if you don't experience it, it can be hard to relate. For me it's something like a buffer underrun. I get bored and frustrated in between words if they come too slowly.

Another analogy might be trying to ride a bike too slowly. It's very difficult not to wobble extremely and fall off. It takes more effort to ride slowly. My mind can't wander with slow speech because it takes great effort and attention to maintain ideas in active memory long enough to make it through sentences. I am better able to have branching thoughts while keeping up with the thread of speech when the speech is faster.

I frequently drive with audiobooks on and it is usually not possible for me to follow along without a speed boost of at least about 25% or so.