It interferes with my thinking mode - right hand on the mouse and probably scrolling in docs on the other screen, left hand stroking my beard.

More seriously, I'm just the same with hjkl and vim - cool if it works for you, when I am scrolling through I don't need to be efficient (which is highly subjective anyway), I need to have it how I like it.