> If I'm on the sidewalk and take a step toward the road, should it make the car jerk?

No. That's my point. Take less drastic measures earlier, and only escalate when you have to. That's how I drive, and a self-driving car can do the same.