For me as a long term Emacs user, started using it 1994ish, it is not an issue. I don't mind the occasional minor lag. In my usage it is rarely in a range where it actually makes me wait.

Emacs has a certain feel to it. Hard to put the finger on, but the movement of the cursor and the scrolling, it just feels nice for me. Like a well worn favourite T-Shirt.