You're just vastly underestimating the stability four rotors bring. They're not flying in a vacuum. There are gusts of wind on top of actually moving the copter where you want him to go.

That can't be reasonably compensated by unless you use a processor which requires way more energy then the rotors ever will.