Counting buttons is no less distracting than a quick glance away from the road. If you don't have time to glance at a screen, you shouldn't be messing with physical buttons either.

The trouble starts when the UI is designed poorly enough to require more than just a glance.