News for hackers. Stuff that matters.
Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @01:00PM (#12017809)

  However, when the second accelerometer failed, a latent software
  anomaly allowed inputs from the first faulty accelerometer to be
  used, resulting in the erroneous feed of acceleration information
  into the flight control systems. The anomaly, which lay hidden for a
  decade, wasn’t found in testing because the ADIRU’s designers had
  
  never considered that such an event might occur.
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An important lesson. Assumptions can kill.
Re:Automated to Death (2009) by xanderstrike on Friday July 01, 2016 @03:18PM (#12018950)
Here's my theory. The engineers assumed it would never occur because they fail rarely and if one failed you'd replace the unit.

Then it got into the hands of the airlines and they said, "You mean it'll run with one failed accelerometer? Then we don't need to replace it when one fails."

Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @03:43PM (#12019120)
If they were even aware that it'd run with one failed accelerometer.

Aircraft maintenance outside the US military (Air Force in particular) terrifies me. While it's not perfect, the USAF essentially rebuilds most of its fleet every X years. There's a reason they're flying planes, successfully, from 60 years [ago]. [EDIT: Missing a word.]

The airline industry does not do this sort of depot level work. Instead, they discover a crack, the put on a piece of sheet aluminum and "patch" it. Wash, rinse, repeat. 10 years later the aircraft is still flying but at greatly reduced fuel efficiency because, like an overweight, middle-aged man, it's gotten a bit of extra weight on it all the damn time.

This is what they do for structural maintenance. I don't even want to imagine what happens with electronics and other subsystems. They're literally willing to cost themselves millions of dollars a year in extra fuel consumption (across their fleet), rather than spend the money to do real maintenance on it.

Re:Automated to Death (2009) by mahyarm on Friday July 01, 2016 @06:13PM (#12020075)
So what does the USAF do when they find a crack? If they do rebuilds it sounds like it takes a lot more time to do the same fix.
Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @07:13PM (#12020368)
They monitor and patch it but eventually the plane gets stripped down to basically the frame and rebuilt.

Their patches are treated as temporary patches. The airline industry has a bad habit of making them effectively permanent.

Re:Automated to Death (2009) by ArkyBeagle on Friday July 01, 2016 @01:43PM (#12018138)
Somehow, "never considered" seems a different thing than "assumed it would never happen".
Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @02:03PM (#12018316)
My reading is that the assumption was: There will never be two faulty sensors in flight.
Re:Automated to Death (2009) by ArkyBeagle on Friday July 01, 2016 @02:51PM (#12018717)
Understood. It's just ambiguous. Being nauseatingly pedantic :), it just seems that the probability of not thinking of it is higher than thinking of it and dismissing it.

I just have a failure-fetish :) What ya gonna do?

Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @02:57PM (#12018773)
That's fair. And see my comment in the parallel thread. In that situation, I think the original tester just never conceived of the possibility that there would be a failure with reporting of fires when two systems which functioned individually were run together. So the procedures didn't have the situation defined in the test plan. It was in beefing up the test procedures that the error was discovered (and others).
Re:Automated to Death (2009) by ArkyBeagle on Saturday July 02, 2016 @02:21AM (#12021610)
"The original tester" was probably working from a Byzantine morass of requirements.
Re:Automated to Death (2009) by Jtsummers on Saturday July 02, 2016 @09:01AM (#12022314)
Surprisingly not. I got to read all the original documents. They were unimpressive in every way.

I had the pleasure of putting everything into a proper requirements tracking database (a thing that company actually did get right by the time I got there, moved away from word and excel based document systems). It was all pretty straightforward and a very simple system in the scheme of things. Fundamentally the sensors all worked fine. There was a box which collected all the sensor data and that was what was failing.

Re:Automated to Death (2009) by marcosdumay on Friday July 01, 2016 @02:08PM (#12018360)
That's going in the direction of proving software correctness. What, for software that interacts with the real world, and must take real word context into account stops being viable really fast.
Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @02:34PM (#12018585)
Yes, we should go in that direction on anything safety critical.

This is a real world situation that I encountered and led to me leaving the company, they did correct it after I left, however:

Fire detection system in multiple areas of a plane. If area A reported a fire before area B, then area B's report would be ignored. Reverse the situation and area A's report is ignored. This system, in particular, discovered fire/overheat early so that corrective action could be taken. By not making the second report it virtually guaranteed a fatal consequence should the (admittedly) very low probability event occur.

Also in this situation, like the aircraft in the article, there were a very small number of areas and sensors. Full, automated testing was entirely possible of literally every combination of sensor fault condition, ordering, and delays (to simulate "faulted" hardware, that is a sensor that reports a fire but isn't backed up by the other 2 sensors in its area). I calculated it at the time, the full series of automated tests could have been executed in a week, entirely reasonable given the lives we were responsible for.

However, we were not given a budget for this (until after I left), and so we were stuck with manual testing. Which drove the costs and time to an extreme amount (also made some testing effectively impossible, "flip this switch, flip this other switch after 3 but before 3.5 seconds"). The entire setup was a fucking joke. And lives depended on it.

On top of this, the software was a complete clusterfuck of shared variables used as temporaries across multiple subroutines, values getting erased (probably causing the problem that I discovered) when stepping through the piss-poor attempt at a state machine.

When hundreds of lives can be lost by the absence of a single, sensible, test, I'd gladly accept an extra couple million up front on a project, rather than deal with the emotional consequence of knowing my responsibility in their deaths, paying out millions to their widows and orphans, and possible civil and criminal consequences for the PEs involved.

Re:Automated to Death (2009) by marcosdumay on Friday July 01, 2016 @03:56PM (#12019204)
That's still depends entirely on the number of variables you must take into account.

You are ok with an extra million of up-front cost, and that's reasonable. With a little bit of added complexity, that cost reaches an extra billion. Is it still ok? With a little more, and it's now a trillion.

This is a really hard problem.

Re:Automated to Death (2009) by Jtsummers on Friday July 01, 2016 @04:03PM (#12019246)
And that's where risk analysis comes in. But modeling the system, to a certain reasonable scale (read: feasible with respect to time), and sane programming practices can seriously mitigate the errors these systems have.

I'd be willing to wager that the faulty system discussed in my quote had a set of variables like this at the top:

  int t1;
  int t2;
  int t3;
Which are all used by various functions later on, obliterating the value the others set. And someone forgot that they were supposed to be temporary. I'd even put $100 on it (I'm poor, ok?).