>sort of small controller that can act as a pointer and has a button or two is superior in every way.

Something named after a small rodent that we use already. And a monitor that we use already. Then you are cooking. You've invented the desktop pc.