Re:Tesla removes parking sensors, the results are predictabl...
by bumby on Tuesday April 04, 2023 @02:41PM
(#35444161)
In various times in my past I've been a private pilot, airframe mechanic, flight-control-computer engineer, software quality engineer, and aerospace safety manager. I've even worked with Boeing. So I am quite familiar with these concepts.
>It wasn't really a risk. It was to make it behave the same.
Hard disagree here. The fact that it did not behave the same demonstrates there is a real risk. That risk could have been mitigated in various ways (e.g., engineering via hardware or software, administrative via training etc.) but they did not. You downplaying the risk as not credible is the same mistake.
>Your thesis that using software to run it as some unusual risk is simply dead wrong.
My thesis is that they didn't know how to effectively characterize the software risk because, as you point out, software risks are different than the risks of mechanical failure. Software doesn't wear-out or display time-variant hazard rates like mechanical systems. Rather, it incurs "interaction failures." The prevalence of these failures tends to grow exponentially as the number of systems that software touches increases. It's a network effect of using software to control and coordinate more and more processes. Which is why we need to shift our thinking away from the mechanical reliability paradigm when dealing with software risk. Nancy Leveson has some very accessible write-ups on this idea.
So what does that mean in the context of Boeing and, in a broader sense, Tesla? Boeing did not capture these interaction risks because they had an overly simplified idea of the risk and mitigations. They did not capture the total system interactions because they were myopically focused on the software/controls interface. They did not capture the software/sensor interface risk (even though their HA identified that risk). They did not capture the software/human interface risk. They thought it was a "simple fix". Tesla, likewise, is trying to mitigate one risk (supplier/cost risk) with software. FTA seems to implicate them in not appropriately characterizing the new risks they introduced. I'm saying that is a result of a faulty design philosophy that downplays (or is ignorant of) those risks.
>It wasn't really a risk. It was to make it behave the same.
Hard disagree here. The fact that it did not behave the same demonstrates there is a real risk. That risk could have been mitigated in various ways (e.g., engineering via hardware or software, administrative via training etc.) but they did not. You downplaying the risk as not credible is the same mistake.
>Your thesis that using software to run it as some unusual risk is simply dead wrong.
My thesis is that they didn't know how to effectively characterize the software risk because, as you point out, software risks are different than the risks of mechanical failure. Software doesn't wear-out or display time-variant hazard rates like mechanical systems. Rather, it incurs "interaction failures." The prevalence of these failures tends to grow exponentially as the number of systems that software touches increases. It's a network effect of using software to control and coordinate more and more processes. Which is why we need to shift our thinking away from the mechanical reliability paradigm when dealing with software risk. Nancy Leveson has some very accessible write-ups on this idea.
So what does that mean in the context of Boeing and, in a broader sense, Tesla? Boeing did not capture these interaction risks because they had an overly simplified idea of the risk and mitigations. They did not capture the total system interactions because they were myopically focused on the software/controls interface. They did not capture the software/sensor interface risk (even though their HA identified that risk). They did not capture the software/human interface risk. They thought it was a "simple fix". Tesla, likewise, is trying to mitigate one risk (supplier/cost risk) with software. FTA seems to implicate them in not appropriately characterizing the new risks they introduced. I'm saying that is a result of a faulty design philosophy that downplays (or is ignorant of) those risks.