I don't think that's part of "visualization" itself. I don't visualize a scene, but I still can take a long time if it's interesting or has a lot of detail. I remember reading the Lord of the Rings books and it has really long descriptions of the scenery and sometime it took me a very long time to read it, just so I can get all the details. In other parts I read over it as fast as I could to get on with the story.

I imagine that's the same for "both sides".

Perhaps, but for me, it's more like having to slow down to let the scene play out in my mind. But that's really interesting to hear about your experience.