Re:Why autonomous trucking is harder than autonomous rideshare
by Edman274 on Monday January 15, 2024 @02:49AM
(#38998367)
I don't disagree that a calculator and a car are different things, but the important thing for the analogy is that the wider society expects the same thing of them. When John Q Public thinks about self-driving cars, he thinks to himself "How hard can it be? Drive the speed limit, stop at stop signs and traffic signals, go when lights turn green, stop at obstacles and stay between the lane dividing markers. If these vehicles can't do that then what good are they? Why risk letting those things on the road if I can't even trust them to do that?" Obviously you and I both know that the base things like "recognizing a stop sign" required decades of research to become reliable, but like, back in the 1960s before the problem had work started on it, even PhD holding computer scientists were thinking that object recognition was basically a done deal.
What I want to keep reminding the software developers with consequentialist ethics is that the entire rest of the world operates under a totally different mental framework than the techno-utopian one. In that mental model, a self driving car is an appliance and it will be judged by the standard of all appliances, which is that any catastrophic failures that happen during their normal operation are evidence of a defective, untrustworthy product. The problem they're trying to solve and its complexity has little bearing in that judgment. What incentive does anyone have to be generous and forgiving about potentially fatal errors when you're not taking the consequentialist viewpoint for granted? That's the thrust of my observation.
An autonomous vehicle has an uncountable number of operating modes, and it is not feasible (perhaps not even possible) to test it in all possible conditions and states. Even if you could, doing so for (say) every single software change would take years each time.
Maybe that does mean that this is a fool's errand, and we just shouldn't be building autonomous cars, at least not until we have AGI that can think and act faster and with better judgment than a human.
I personally do think that "better record than a human driver" should be sufficient (perhaps with some significant, TBD margin; 0.1% better is probably not enough), but I accept and agree with your toplevel comment that sort of thing won't fly in the real world. The bar is really more like the self-driving car has to avoid making the specific kinds of mistakes and illegal/unsafe acts that a human driver would do (all while not creating new classes of mistakes that a human driver would not make), and, on top of that, be better than a human driver in situations where crash would not be deemed that human driver's fault.