On traditional pedals yes, you can make pedals/levers that would just hold their position until altered by the passenger. The core idea is that the feet should be selecting the speed.

Yeah no, I get that. But as you hit the bump, your foot would leave the pedal/lever, and when you come back down again, your foot will press against the pedal/lever, increasing the acceleration as you inadvertently pressed it down.

Hmm just had a brain wave, going to see if I can make it and show you guys instead.