The problem is, you'd be hitting all these pesky corner cases, with some of them requiring active avoidance - downed power lines, water on the road, vehicles on fire, other traffic participants. My guess, it is hard to engineer a system that could handle the long tail of making decisions under uncertainty with "purely physical approach". You'll run out of complexity budget.