News for hackers. Stuff that matters.
Re:Boeing Withheld Information on 737 Model, According to Sa... by jakobegger on Tuesday November 13, 2018 @03:14AM (#18439475)
> Automation overriding pilot inputs during sensor failure is a recipe for disaster.

In 2009 an Airbus A330-200 crashed into the Atlantic ocean killing more than 200 people because a confused pilot kept pulling the plane up, despite loud stall warnings from automatic safety systems [1]. In the discussions of that event, one commenter asked, "Why did [the plane] not override the pilots' inputs and force a pitch-down?" [2]

Designing safety critical systems is difficult. If the automated system fails, the pilot needs to be able to take over manual control. But at the same time, if the pilot does something stupid, the automatic system should prevent that.

I'm not sure how to square that.

[1]: https://www.telegraph.co.uk/technology/9231855/Air-France-Fl...

[2]: https://news.ycombinator.com/item?id=8452080

Re:Boeing Withheld Information on 737 Model, According to Sa... by mnw21cam on Tuesday November 13, 2018 @06:16AM (#18440153)
One of the confounding elements of that particular crash was the fact that when the angle of attack increased above a certain (crazy) amount, the stall warnings went away, due to the inability of the system to cope with an angle of attack that high. This confused the pilots into thinking they were going "crazy fast", when actually they were going "crazy slow".
Re:Boeing Withheld Information on 737 Model, According to Sa... by richardkmichael on Tuesday November 13, 2018 @03:00PM (#18444340)
Yes, I point out AF447 as a perfect and horrific example of a problem which is really HCI (human computer interaction) / UX.

The lack of stall warning meant two different things: "not stalling", and "stalling so extremely it can't be sure, so don't report it"; as you say. I wondered if there should be a different sounding alarm for "can't determine". Or, the pitch of stall alarm could rise, such that pilots could tell "which side of the stall" the plane is on, so that entering stall from one side would side start with a very high pitch or very low pitch. (I'd guess there's a better way..)

So, in this case, it's more subtlety more tragic than you describe, because the pilot did correct the AoA, and nose down, but when he did that, the stall warning resumed (albeit from the extreme other "side" of the interval), confusing him: no alarm, alarm resumes, so he thought he "re-stalled" it and he pulled up again. This kept him mentally "stuck" in the extreme stall. They needed to "pass right through" the stall warning window, and come out the other side. They had sufficient altitude for this to succeed.

Re:Boeing Withheld Information on 737 Model, According to Sa... by AstralStorm on Tuesday November 13, 2018 @03:43AM (#18439579)
This is equivalent to a confused deputy variant of a Byzantine failure.

Since you already have allegedly redundant pilots and alarm systems and hopefully sensors, the remaining part is figuring out which failure detector has ultimately failed to work and why. Also if redundant pilots are failing, it's a systematic failure in training, fatal to airlines and whoever designs training simulators.

The most potentially fatal to Boeing scenario is a systematic failure or negligence in terms of redundancy. Hampering pilot training is bad enough. Another potential failure point is a software issue which means negligence in safety critical software engineering.

Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @08:34AM (#18440781)
I disagree. Flying a plane is pretty much a simple state machine with very well defined states. We have hundred thousands of flight hours available and we can also build a very realistic simulator which is used extensively to train pilots. It is should not be a challenging task to build a reliable auto-pilot and in fact we have already one that covers most of the aspects of flying. On the top of this, most of the flight related incidents are human errors.
Re:Boeing Withheld Information on 737 Model, According to Sa... by Scramblejams on Tuesday November 13, 2018 @04:01PM (#18444866)
You have been misled concerning the accuracy of simulators. "All models are wrong, some are useful" is a helpful quote to remember here. Certified sims are typically very accurate when it comes to avionics and systems. They are often much less representative where vehicle dynamics are concerned.

Even the most heavily analyzed aircraft can produce surprises once they actually take to the air. Look up "Super Hornet uncommanded wing drop". Or look at dynamic scaling tests (https://www.nasa.gov/pdf/483000main_ModelingFlight.pdf) which uncovered two spin modes in the F-15 that were previously entirely uncharacterized. (Not sure if the linked doc covers that instance specifically -- I learned this from speaking with the study PI. And no, most aircraft programs don't get to do dynamic scaling tests.)

All this is to say that a variety of potential non-linear behavior is lurking inside any given airframe, and all the wind tunnels and CFD in the world will merely allow you to model most of the useful operating conditions, but certainly not all of them.

Simulators, just like autopilots, are useful for finite input ranges. If you leave those input ranges, your simulator will precisely and repeatably model a fantasy, and your autopilot will disconnect.

Source: Short career as an aerospace engineer.

Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @06:45PM (#18445964)
And how do you train pilots for these unexpected non-linear events? What stops you to train auto-pilot the same way?
Re:Boeing Withheld Information on 737 Model, According to Sa... by Scramblejams on Tuesday November 13, 2018 @07:10PM (#18446116)
The point of my post was to say that these unexpected events will not come up in your simulated environment, so you simply will not have the data needed to train the autopilot.

The human, by and large, figures it out. We generally accept that risk. But how will the autopilot react to the unexpected event for which you haven't trained it? Who knows?

Re:Boeing Withheld Information on 737 Model, According to Sa... by noselasd on Tuesday November 13, 2018 @12:10PM (#18442612)
It doesn't sound that plausible that we can account for all the various things that can happen to a plane in something as complex as the earths atmosphere with a "simple state machine" . Weather can do really wacky things.
Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @02:25PM (#18444002)
You are saying that it is impossible to implement auto-pilot? How does human pilot fly something planes? You think that human mind can deal with infinite states?
Re:Boeing Withheld Information on 737 Model, According to Sa... by nradov on Tuesday November 13, 2018 @03:01PM (#18444352)
Yes the human mind can deal with an effectively infinite number of states (and let's not get into a pointless pedantic discussion about the true meaning of "infinite"). Until we figure out how to build a true AGI then no autopilot will be able to do that.
Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @03:36PM (#18444660)
No it can't. Flying an airplane on Earth is a very limited exercise in every possible dimension and it can be easily automated without AGIs.
Re:Boeing Withheld Information on 737 Model, According to Sa... by nradov on Tuesday November 13, 2018 @03:41PM (#18444692)
You're simply wrong and have no idea how flight control actually works. In particular failure modes aren't simple at all; real machines can break in a huge number of ways and no one could possibly predict them all. Recommend you take some actual graduate level aeronautical engineering courses and then reconsider the issue.
Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @06:43PM (#18445957)
I am not arguing about how machines can fail, I am arguing for that computers can do things that humans can. Usually with higher precision and much more reliable.
Re:Boeing Withheld Information on 737 Model, According to Sa... by nradov on Tuesday November 13, 2018 @08:55PM (#18446755)
Machines can do some things with higher precision and reliability than humans. Coping with unexpected aircraft failures aren't currently among those things.
Re:Boeing Withheld Information on 737 Model, According to Sa... by Scramblejams on Tuesday November 13, 2018 @04:10PM (#18444927)
noselasd ain't wrong. Autopilots can and do disconnect when the weather conditions exceed their ability. Happens all the time, though probably not often in large airliners because the big carriers have people dedicated to watching the weather and keeping their pilots out of trouble. Your state machine will have to do that, too.
[dead]
Re:Boeing Withheld Information on 737 Model, According to Sa... by StreamBright on Tuesday November 13, 2018 @02:26PM (#18444010)
I know this is why do not have auto-pilot.