This assumes the skill variance among pilots is pretty low. Average not in skill (I don’t think there’s a way to rate or rank pilots) but in specialised Boeing plane specific training. The problem with Boeing’s assumption was that pilots would have specialised training on that model, and the regulators seem to be trying to remediate that.

If the skill variance is pretty low then you might as well use test pilots. Their skill will hardly be better then the worst of the worst.

I don't think there is actual data on skill variance among professional airline pilots so I rather they just force testing on groups which are statistically more significant to get into crashes.

But aren’t test pilots specially clued on to the plane, the features they’re testing and how to react in problem situations? That’s like testing car airbags with stuntmen who know how to crash without getting hurt anyway.

Keep in mind that even with test pilots briefed on the behavior, one of the three ended up losing the plane during testing of single event upsets of the Flight Computer.

https://www.seattletimes.com/business/boeing-aerospace/newly...

That absolutely does not follow. Test pilots are prepared and trained for the extremes of control. It's a totally different skillset.

The point of the test flight is to push the plane as far as it will go. Practically all drivers, regardless of skill level, will lose control if you have them hydroplane a car. Race drivers will not. With a couple advanced driving classes suddenly most people will be able to handle a slide much better. There's a gap in experience, not talent. There is no such gap between any two normal drivers.

I don't believe the wording means arithmetic average.

It's more akin to the reasonable person test in law. Meaning, a reasonably well trained pilot that is not considered significantly more expert than those around him or her.

Just a side note from a law nerd. 'Reasonable person' doesn't mean 'typical' or 'ordinary' person, it means 'person who is amenable to reasoned argument.'

Suppose you make a proposition to a person and they reject it based on some fallacious argument. You, an intellectual, kindly point out the fallacy in their objection. A reasonable person is one who takes this in and reconsiders your proposition. An unreasonable person is someone who doubles down and either offers different but equally fallacious arguments (if they just hate your proposition and are looking for an excuse to reject it, but wish to conceal that) or refuses to acknowledge their original fallacy (due to embarrassment, stupidity, or perversity).

So, like, an NPR listener vs a Fox News viewer?

I agree. Though I would also feel pretty disheartened if I got a call from them after that decision was made -

"Hey, we need someone of your skill level to come in and fly some test planes"

"...okay"

Call it training not testing and the morale problem probably goes away.

Naw, you have everyone try to recover the plane in the simulator.

The Max needs to ensure that 80% ("average" is too low a standard) of tested pilots recover the plane.

The other 20% take remedial classes.

Imagine the job postings for such positions ...

You would word it differently, such as "Max 2 years of experience".

You could just select all applicants whose name is Max or Maxine, to get statistically average pilots.

While I don't disagree, at least moving from test pilots to average pilots would be a big improvement. And, in practice, might mean that you have to simplify and streamline the UI just as much as for the minimum viable pilots. But, even if not, at least average would be way better than test pilots only.

"Simplify and streamline" is exactly not the solution here, in fact the issue was that the relevant information was invisible and hidden from pilots, not that there was too much. Not everything in good design needs to end up looking like a Fischer-Price toy.

The main problems from my limited understanding were: pilots didn't know the subsystem existed; they didn't know it had activated; they didn't know what it was doing; they didn't know how to turn it off.

Additional instrumentation could have addressed all these issues but would have been more expensive than what they actually did.

True, but too much instrumentation is also a problem.

I'm into synthesizers, which run the gamut from bleep-bloop toys to movie-soundtrack in a box.

Mostly, synths are marketed like 'Not only can the Omnicron Neutron Wrangler produce any sound imaginable, it can produce more of them more loudly than the competition and audiences will surely be impressed by your mastery of its mind-bogglingly vast control surface.'

But the synths which end up being well-loved classics are not the ones with the biggest range or the most controls, they're ones with a decent range which is designed and calibrated well enough that it's hard to get a bad sound out of it - meaning they don't have too many abrupt phase transitions or nonlinear feedback loops that could cause you to suddenly lose control and turn an interesting complex tone into noisy garbage.

That's true, but the design constraints are very different. If your synth had a system in it that could fail, and another system responsible for monitoring that first system, and potentially overriding it, and if that system failure required you to take immediate and very specific action to prevent your synth from turning into a fireball, you'd want probably want a more or less one-to-one mapping between those subsystems and the instrumentation that you're looking at.

The well-loved classics you describe sound more like making a plane that's aerodynamically hard to lose control of... which is kind of a holy grail, but the MCAS design came after all of that was fixed by earlier design decisions.

I suspect the test pilots represent the <1% of pilots who are 10x skillful as a typical pilot.

Depends on the distribution

You're technically correct, but for the metric "human pilot skill" I think the median and average probably align fairly well.

I would guess that the skill distribution of licensed pilots is asymmetrical, if only because the tail end of bad pilots are not allowed to fly.