Re:What Self-Driving Cars Tell Us About AI Risks
by jaclaz on Monday July 31, 2023 @05:49AM
(#36940460)
With all due respect, it seems to me like your simple explanation explains very little.
There is no way (besides kids having fun) that a human driver would intentionally hit a cardboard box found in the middle of the road, full or empty as it might be, and as well an automated system will brake or avoid the cardboard box.
But the point is that even when there is no cardboard box anywhere (nor pedestrians, nor cars, nor anything) these "phantom breakings" happen, example:
https://theintercept.com/2023/01/10/tesla-crash-footage-auto...
It is obvious that something went wrong in the sensors, in the tramission/processing of the data or somewhere there is a bug in the algorithms, but (AFAIK) nothing specific (and reproducible) has been found, in this sense "The cause of such events is still a mistery." seems to me correct.
> One failure mode not previously anticipated is phantom braking. For no obvious reason, a self-driving car will suddenly brake hard [...] The cause of such events is still a mystery.
There's no fundamental mystery about phantom braking. A self-driving car (or ADAS car with auto-braking) is 'simply':
1) Observing the world around it using whatever senses it's equipped with
2) Predicting the future state of that world and itself based on those observations, and
3) Controlling its output actuators as required to achieve desired outcomes and/or limit undesirable outcomes.
This is nothing controversial, this is the basic structure of any autonomous agent, from a toilet cistern to a kung fu master. So if a car 'phantom brakes' then it's perceived what it believes to be an imminent collision, and acted accordingly.
For simple radar-based systems, they can't really tell the difference between different vehicles or different lanes, so they guess based on the car's speed and any other speed readings they get with an intensity above some cut-off. That's why they don't work at low speed, because they can't tell the difference between other vehicles and stationary objects. These systems work okay-ish but there are tons of scenarios where they fail, and the system misinterprets its very vague sensory input as "car going much slower than us in front of us" and hits the brakes.
For more advanced systems that are doing vision etc. they're identifying objects and predicting how they'll move, and planning accordingly. This is usually better but still a Hard Problem (consider driving past a crowd of pedestrians, are any of them going to step out in front of you? Or consider a cardboard box in the road, is it just a box and safe to hit, or is it full of bricks?) so it's not surprising that current systems sometimes get it wrong. So do humans.