It's faster with the keyboard because your hands don't leave the keyboard. There are so many ways to jump around quickly with the keyboard. In emacs I just use searching to jump around. Selecting is easy once you've learnt to navigate. The keyboard is an instrument you can get really good at using. You can't get good at the mouse. You're as good as you'll ever get already.