Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too.
https://www.youtube.com/watch?v=ufNNuafuU7M
This was not a hard fail for the AI.
(Or, more charitably, "oops, somebody forgot that object persistence is a thing" does not excuse the result)
It’s clear that a) the road is well lit and b) visibility is far, far better than the Uber video would suggest.
An ordinary human driver would have seen Elaine & taken evasive action. This death is absolutely Uber’s responsibility.
Looks like this was a hard fail for the AI then. We can say with better than 90% certainty that a human would have saved the situation, probably would have stopped or avoided easily. My mistake.
We can reasonably assume that pja is aware of the existence of abysmal drivers and fatal crashes that should not have happened. I doubt their intent was for "would" to be interpreted as "100%".
Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-de...
And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17
And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU
And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716
These accidents are 100% due to the decision to use a janky vision system to avoid spending $2000 on lidar; and that janky vision system failing.
https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg
The car had LIDAR.
"the car detected the pedestrian as early as 6 seconds before the crash" [...] "Uber told the NTSB that “emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior,” in other words, to ensure a smooth ride. “The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.”" [1]
'Oh well it was very dark' is not a factor in the crash that killed Herzberg
[1] https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detect...
There are many other cheaper LIDARs, even in the Velodyne lineup, but they are less capable.
The vehicle was equipped with both radar and lidar. The victim was detected as an unknown object six seconds prior to impact, and at 1.3 seconds prior to impact when the victim entered the road the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this.
Why would the system notify the driver that emergency braking was required instead of simply braking?
Just adding this if you think we should somehow give Uber the benefit of the doubt here. They released footage from a pinhole dashcam sensor that is not used by the system, knowing fully well it would be pitch black and send the ignorant masses into a "she came out of nowhere!" chant.
With the sensor array available to it the car should have done better, no question.
But to make the claim "fails harder" I would be looking for a clear cut situation where a human would almost definitely have outperformed the AI.
Human eyes do have miraculous dynamic range so we would likely see more. Can we say with 90% certainty that a human would have saved the situation?
Yes, given the misleading nature of the dashcam video I think we can. This was not a pitch dark road lit only by headlights where an obstacle "appeared out of no-where". This was a well-lit main street, with good visibility in all directions. An ordinary human driver would have had no problem identifying Elaine as a hazard and taking the appropriate avoiding action, which was simply to slow down sufficiently to allow her to cross the road.
The backup driver in the car was apparently looking at their phone or some other device and not watching the road at the time.
"According to Uber, the developmental self-driving system relies on an attentive operator to intervene if the system fails to perform appropriately during testing. In addition, the operator is responsible for monitoring diagnostic messages that appear on an interface in the center stack of the vehicle dash and tagging events of interest for subsequent review"
She was looking at a device, yes, but not her phone.
Uber put one person on a two person job, with predictable results.
https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg#Distr...
The rest of my point still stands though.
Moreover, try this dashcam video: https://youtu.be/typj1asf1EM It's 10 seconds long, and makes the pedestrian look almost invisible except for the soles.
However, when I took that video, both the crossing pedestrians were clearly visible, not vague shapes that you only notice when you're told they exist. So much for video feed fidelity.
I think every human would slow down if they see things that they cannot explain. An AI will not.
It's basically the same problem as when an image recognition AI is 99% sure that the photo of your cat shows guacamole.
Current AIs do not have a concept of absolute confidence, they only produce an estimate relative to the other possibilities that were available during training. That's why fully novel situations produce completely random results.