Typical code cowboy mindset: expects all of the glory, none of the responsibility. Kindly piss off and don’t come back until you’ve learned proper systems design.

https://en.wikipedia.org/wiki/Fail-safe

Self-driving software might find its way around a controlled test track okay but is nowhere near fit to be in control on public roads. Safe way to test it the is to have human drive and have the software simulate the same drive, then compare the two for discrepancies. Rinse and repeat until the software consistently equals or betters human decision-making. Then you can consider putting the software in charge.

Programmers are supposed to be the technical experts in the room. So if they aren’t pushing back against ignorant uninformed hubristic Management’s bad decision-making and making it right, how the hell do they think their software will do any better? Fucking Worse than Useless, the whole cowardly bloody bunch.

I don't think you understand the basic concept let alone how the self driving systems are comprised and how large software development projects work. The uber car wasn't self driving. It required a human occupant to watch the road at all times. That is certainly not on the developers.

You appear to be laboring under the gross misassumption that the human is there to take control when the machine fucks up. They are not.

They are there to take the blame.

You cannot realistically expect a human spectator to switch modes at zero notice and save the machine from a deadly fuckup that the machine has already put them into. That’s not how human minds work. We just don’t bootstrap that fast. Jebus, we’re bad enough at salvaging bad situations we’ve knowingly manoeuvered ourselves into while already fully engaged in command and control mode.

At least when you—the human pilot—fucks up, you already know the decision chain that got you there because it is your own. With the machine you have first to determine it has gone catastrophically wrong, then determine how it has gone wrong, and finally calculate and execute the recovery strategy before…oh, whoops, too late: you just wiped out all the executive bonuses for this quarter. Also, there’s a blood streak on the street.

As for how large software development projects work, I think the trail of corpses already testifies more than suf as to how they don’t. Industrial institutionalization of incompetence is no defense, and any “professional” who hides behind it can go get fucked.

Exactly. Other companies knew that kind of attention and reaction was beyond human capacity, which is why they had two safety drivers. Which I'm still not sure is sufficient to compensate for the other sorts of negligence Uber staff indulged in here.

As much as I love the web, I think the move-fast-break-things ethos, which is arguably useful for startups doing who-cares-if-it-breaks things like social web front end tweaks, has been absolutely terrible for the industry more broadly. I have friends who make excellent money just sweeping up after the elephant parade of hotshots, solving infrastructure and code issues written by people who want to get paid like professionals without acting like ones. I'm glad for them, but the waste is maddening. And that's before we get to the body counts of places like Facebook and Uber.

That is not what the NTSB said.

Responsibility is not zero sum. The safety driver is responsible. But so are the people who sent out robot cars with a single safety driver, as they knew or should have known that paying attention in low-interaction situations for many hours in a row is not something humans reliably do.

The Uber managers and execs are also responsible, in that they set up the system that led to needless death.

But none of that absolves the programmers, some of whom did things that they knew or should have known were dangerous, and who did not make sure that the system they were committing code for was set up for proper safety.

Your notion that the only job of programmers is to meet the spec is one I deeply disagree with. We're not Amazon warehouse workers, desperate for a job and blindly following whatever orders come our way. We're highly paid professionals whose job is to understand what we're building and what effects it has. I think that's true for any sort of coding, but I believe it's very obviously true for life-critical systems. If we can't handle the responsibility, we shouldn't cash the (quite large) checks.

I agree, partly. They were testing the vehicle, and having a human in the driver's seat to take over in the event of a problem is a major component of that testing. That person likely became complacent in their duties due though, likely because the software seemed to work well enough.

The failure here was to expect someone to stay alert for a rare event for long periods of time. They probably should have had a more active task that would have kept them more alert.