Something like the radial keyboard used in Steam's big picture mode could be really useful here. It would reduce potential error surface by quite a bit.