News for hackers. Stuff that matters.

A comment on Boeing wants FAA to exempt MAX 7 from safety rules to get it in the air. Parent comment

Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by aeturnum on Friday January 05, 2024 @05:05PM (#38885615)
There was a great paper[1] I read about the human components in complex technical systems which argues that one of the roles of the human is to take the blame when the entire system fails. This does real valuable work for the companies involved and helps them avoid needing to answer the most uncomfortable questions.

[1] Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction by Madeline Clare Elish

[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by jsight on Friday January 05, 2024 @08:17PM (#38887295)
The flip side to this is that every system involving software seems to inevitably devolve into a situation where the human is expected to no longer be responsible.

Oh, you floored it while the car was pointed at the wall? It has cameras, why didn't the car disable the go pedal?

This is happening more and more with cars, and it seems inevitable that it will happen in other spaces as well, as software is expected to protect us from ourselves.

[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by underwater on Friday January 05, 2024 @09:14PM (#38887674)
There is a big difference between software being the safety net for humans, and humans being the safety net for software.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by jsight on Friday January 05, 2024 @10:01PM (#38887950)
In the extreme, yes. But it looks like it is actually a continuum, with a shockingly blurry line forming the threshold.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by jxf on Friday January 05, 2024 @07:06PM (#38886807)
"Moral crumple zones" is an incredible description.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by jacquesm on Friday January 05, 2024 @06:34PM (#38886571)
This goes for self driving cars just the same.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by TaylorAlexander on Friday January 05, 2024 @06:46PM (#38886663)
Yeah it leads to real problems for fully driverless cars, as there is no longer a human to blame.

Also note that blaming the human has been Tesla's strategy for avoiding responsibility on their software drive system failures.

[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by dev1n on Friday January 05, 2024 @08:19PM (#38887306)
Most car - pedestrian crashes in the USA never end up with the driver at fault. Not sure this will change when cars are autonomous.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by TaylorAlexander on Friday January 05, 2024 @08:48PM (#38887517)
Well I am talking about all crashes, not just car-pedestrian crashes. For example Teslas crash in to other cars and then the company blames the human driver.
[ Parent | on HN ]
Re:Boeing wants FAA to exempt MAX 7 from safety rules to get... by aeturnum on Friday January 05, 2024 @06:36PM (#38886583)
An example used in the paper are the human drivers of Uber self-driving cars - one of which killed a pedestrian in Arizona.
[ Parent | on HN ]