While our eyes can do that pretty reliable, we are organics and get tired - how many hours one can drive until this becomes almost imposible? I had a situation where I would hallucinate and and start believing something is in the street do I did a full stop - nothing happened, but was quite intense. Imagine the other way around - not stopping and hitting something.
A normal radar + some low level ASIC programming would do that without geting tired. My Audi from 2014 is quite good at that and I actually rely on this feature all the time.
Updates are happening around once a month and the decision making is getting noticeably better.
The full system for humans is “vision + brain” and for self-driving its “sensors + planner”.
The Waymo/Cruise philosophy is that since we don’t know how to make the planner human-brain-level, we should shift as much of the load as possible to the sensors, where we have the ability to use things that humans don’t have, like lidar and radar.
To me, Tesla FSD going vision-only is a bet on the progress of AI planning models. If the planner reaches a human-equivalent level, then human-equivalent sensors are fine. Time will tell if this is a good bet, but so far it’s not.
So currently the vision stack is not greatly holding them back.