The brain is able to make itself think that external tools are mere extensions of the body, so it's no surprise that the mouse has been successful. I wonder if the lack of any tactile response in these interfaces hinders them somewhat or if it's all the same for the brain.
The mouse is movement and control, the gestures need you to hold your hands in strange positions and move within a field of view (what the camera can see). An improved design will need to be intuitive.