Problems like that have technical solutions. You phase in updates over time. A hundred vehicles a day for the first week after release, then a thousand a day, then ten thousand a day and so on. That way if there is a serious bug, you find it when it's running on hundreds of cars rather than hundreds of millions.

Just like 346 people flying Lion Air and Ethiopian Airlines became unwilling parts of a "technical solution". No thanks.

Unless you think we should stop having computers in planes, what are you proposing as a solution to that? More to the point, what does it have to do with systemic risk or reducing it by staggering updates?

The comment was criticizing your flippant attitude towards treating humans like beta testers, with the attitude that it's OK to kill a few, just not too many, and dismissal of goatinaboat's initial concern. I don't have a specific answer, because I agree that it's a problem far different than rolling out updates of a new browser version on everyone's laptop. "Technical solutions" acceptable in that scenario simply aren't good enough when the "OS" is speeding down the highway at 75 miles per hour.

Here was goatinaboat's initial concern:

> One bad software update kills... everyone.

That's a fair concern. If you push a software update out to a hundred million cars at the same time which causes some of them to crash into oncoming traffic at high speed, you could have a large number of fatalities by the time you identify that there is a problem with the update.

But phased deployments do mitigate that concern, because it provides time to notice the problem before the bad update is more widely deployed.

That doesn't imply "treating humans like beta testers" because systemic risk is a real concern even when you have thorough internal testing. Even a thousand test vehicles will not encounter the full set of conditions that hundreds of millions will. You would expect to see a flaw that occurs for one vehicle out of 2000 occur 0 times on a fleet of 1000 test vehicles but 50,000 times on an installed base of a hundred million. Having a total of 50,000 fatal traffic collisions in one day is totally unacceptable, but having 10 on a bad day would mark a significant improvement from the status quo.

It's one thing to say "mitigate," which you do here, acknowledging that significant risk still exists. That's very different than the wording of your first comment, "technical solutions." A solution implies the risk is negated.

Aside from the suggestion of beta testing on human lives, you are assuming that your phase-deployment mechanism always works correctly. Even Google has previously botched phased deployments and slow rollouts.

> Aside from the suggestion of beta testing on human lives

There was no such suggestion. You can do thorough internal beta testing and still not uncover every bug that will surface in production, because encountering every possible scenario that will occur in a complex system following mass deployment is only possible following mass deployment, and it is still worth reducing the impact of those bugs.

> you are assuming that your phase-deployment mechanism always works correctly.

It can work correctly 30% of the time and still be 30% better than not having it, because then a serious problem has to coincide with a failure of the phased deployment in order to have widespread consequences. This is why safety-critical systems are layered, so that you don't need them all to work all of the time as long as having all of them fail at the same time is sufficiently rare.

Or that a nice phased rollout over the summer doesn’t fail as soon as leaves fall or it rains

The thing about systemic risk is that it's systemic. You don't have to wait six months for leaves to fall because when it's spring in North America it's autumn in Australia. One place going without rain for months may be common but every place going without rain for months would be unprecedented.