Actually, a lot of things you may perceive as visual are not visual, at least if you imagine "visual" as something I (a blind person) cannot understand.

I do a lot of refactoring, maybe even more than you do, simply because I can't understand functions that have two indentation levels (like two if statements inside each other), or functions that go above 15 lines. I'm saying that I may do it more than you because, unlike sighted people, we can't glance at 6 lines at once; we have to look at the code one line at a time. So, the smaller each section (whether it's an if statement, a loop, or a function, depending on the language itself), the better for me.