You don't necessarily have to do that. I'm adding voice recognition into the game I'm working on right now, for example. It helps that it's a turn-based strategy game to begin with, though.
And there are players that manage to play games with limited or no mobility in their hands at all, with the right accessible controllers.
I don't think I'm doing anything too crazy with it either, so it's a little worrying, since it's happened pretty much every time I've played it lately.
Very cool! I've also found that it's helpful when the game supports different controllers too. I absolutely cannot play games with keyboard & mouse but plugging in an Xbox controller to the PC is completely fine. I'm sure people out there have the opposite experience as me, too.
My game will definitely have controller support also (most of it does now, a few screens are a little spotty I'm planning to fix in the next month or two, and I don't yet have multiple controller support, that may have to wait until later). But it's also fully playable with just clicking a mouse as well.
There was a book I can't recall the name of, in which one of the characters (a human) is stranded on an alien planet. He agrees to a certain type of relationship with his alien benefactor, but doesn't understand the cultural implications of what he's agreeing to. His fingers are lengthened through a medical procedure, so long that he can't hold anything, can't work, can't ever exist on his own again – he exists now only as a thing of beauty, a token for the alien to show off. He spends his days in what I picture as a courtyard in a roman villa, to be shown off to guests. This confers social status on his owner because of the resources it takes to keep a helpless sapient being fed, clothed, and happy. (The last one is debatable.) It is seen as one of the most intimate types of relationships in this culture. I think about this a lot.