Yes that has bugged me quite a bit; I first noticed it when reading his essay on C [1]. But since then I've installed Vimium [2] and I never even touch the mouse anymore.

[1] http://www-cs-students.stanford.edu/~blynn/c/

[2] http://vimium.github.com/