Two things that make this a potential culture issue, as I see it:

1. Software that operates heavy machinery is life-critical. There’s no such thing as “just a bug” in that environment. It requires a different level of care.

2. As I understand it, Cruise was shut down in San Francisco not because of the software error, but because the company didn’t disclose the full details of what happened with the pedestrian that was hit. That’s very much a cultural problem.

If it’s critical enough of a bug that they had to suspend testing immediately then they probably shouldn’t have been doing the testing in the first place

Suspending testing immediately and tracing like this is a process failure is a good sign imo

The process failure was Cruise didn't suspend testing immediately; they waited until they got yelled at.

what bugs are we talking about?

It is alleged that Cruise neglected to provide the DMV with information regarding an incident that led to a pedestrian being badly injured. Other reports claim they willfully withheld information. It appears to be more than a “software bug.”

It's not the crime (software bug), it's the cover up.

I would argue “it depends”.

There are certain professions that operate with public trust. Doctor, lawyer, and engineer are the classical examples. Sometimes those of us in tech get lulled into complacency because we operate under an industrial exemption (ie, don’t require an engineering license) or we work on non-critical systems where we’re encouraged to move fast and break things. We shouldn’t let that attitude spill into safety-critical applications.

If you have a duty to the public to provide a safe product in an ethical manner, the software bug can certainly be the issue.

Actually I'm having a hard time even seeing it as that much of a bug. As I understand the accident, the woman trapped wasn't visible from anywhere in the vehicle or its sensors. Any human driver would likely have done the same thing (pull over after a collision when your vehicle is still drivable). It's a terrible situation for the victim, but it's not something reasonably preventable by safety technology of any kind.

But regardless, that's not what Cruise is on the hook for. They got in trouble because they tried to hide the full truth from regulators. And that for sure is a toxic internal company culture problem.

If a pedestrian gets flung in front of your car and then you no longer see them, wouldn't you be inclined to check whether they might have gone under your car? Especially if you're hearing screaming...

Driver behavior after collisions isn't remotely that rational. People do all kind of crazy things in extremis (secondary collisions caused by post-collision drivers mistakenly hitting the accelerator instead of brake are a whole category of accident!). And in any case (again, as I understand the accident) this was stopped traffic and the Cruise vehicle was hit by another vehicle. In that kind of situation I think a human driver would be hard pressed to know a pedestrian was nearby at all.

From Cruise's statements and news reports, it does not sound like the Cruise car was hit by another vehicle. A nearby vehicle flung a pedestrian in front of a Cruise. The Cruise car stopped on top of the pedestrian, then dragged them as it pulled over. Even accounting for post-crash panic, I find it unlikely that a decent human driver wouldn't have realized that the pedestrian ended up underneath their car.

https://getcruise.com/news/blog/2023/a-detailed-review-of-th...

But what if I belligerently insist that the average human driver would react in the worst possible way? I am trying to appeal to your angriest moments while behind the wheel of a car and get you to believe that your impression of other drivers then is true of most human drivers everywhere, all the time

Then the robot look quite Good, No?

That's not a reasonable argument. The point isn't "The robot is good because you can imagine a human making the same mistake", it's "We wouldn't think this accident was notable at all if it were a human driver, so we at least recognize we're holding the robot to a higher standard".

The truth is that virtually every traffic accident can be prevented on some level by every vehicle involved. People (you included, I assume?) who want to argue against automation in all cases will always be able to find something wrong with "the robot". Always. But that's not the right standard to be applying, because those accidents were going to happen anyway. The question is merely if there will be more or less of them with "robots" at the wheel.

You're right, Waymo and Cruise and Venture Cap are on a moral crusade to prevent driving fatalities, this isn't about high margin robotaxis and delusional hopes of accidentally creating AGI with a CV NN. And I agree with you, what the cruise vehicle did in this case, a human would totally do exactly the same thing. I mean humans are fucking neanderthals, have you ever seen one behind the fucking wheel of a car? They can't go more than 30 seconds without nearly crashing, and they only know how to drive in a 4km^2 zone. I personally know that every time I let my tesla go FSD, I'm only intervening every 90 seconds because of the unreasonably high standards I hold it to. Self driving cars only need to make the right decision like ~80% of the time. That's good enough, it's better than most human drivers.

And by the way, there's a cheaper and more effective way to reduce traffic fatalities, and it uses fewer precious resources, energy and land, emits less carbon dioxide, and stimulates economies but it's some kind of stealthtech from the 1800's. (choo choo)

> I personally know that every time I let my tesla go FSD, I'm only intervening every 90 seconds because of the unreasonably high standards I hold it to. Self driving cars only need to make the right decision like ~80% of the time. That's good enough, it's better than most human drivers.

That's only good enough if the type of the wrong decisions it makes is the same as the ones a human would make. Otherwise the car will seem unpredictable to human drivers and therefore dangerous.

A few baffling Tesla FSD examples:

• Misleading turn signals: https://youtu.be/2VWyaAzwMT0?t=702, https://youtu.be/2VWyaAzwMT0?t=723

• Failure to yield: https://youtu.be/2VWyaAzwMT0?t=1180

• Randomly stopping: https://youtu.be/2VWyaAzwMT0?t=1215

There was a definite bro culture reminiscent of Uber's and we all see all the horrible actions Uber did to try to win / kill it. Unfortunately killing an innocent pedesterian in the process of trying to win / kill it.

Cruise's responses to their many negative events read like Uber's and I expressed my distaste for Cruise numerous times on here because of their bro trying to kill it attitudes (if you're not for robot cars you're out of touch being one example).

I'm not surprised and maybe sharper and more adult heads in GM stopped the program in time saving another poor innocent pedestrian from being mangled by training wheel robot car technology. I.e. The cars all came to a screeching halt when the 5G network it used was clogged up due a concert nearby (wow). It relied on a public 5G network?

Maybe one day robot cars it will be safer then a humans if so it will take decades to more yet is that progress worth these algorithms learning at the expense of more innocent lives?