>> It also noted that employees do not understand how to use the different reporting systems and which reporting system to use and when.
As was noted by the purported insider, re: multiple overlapping systems of record/not-record, Boeing's actual processes themselves are badly in need of overhaul.
This feel like a clear example of where top-down + bottom-up independent read-back verification would have been useful.
I.e. management decides they're going to create Safety Process X using Systems A, B, and C. They do so, then circulate training (top-down). THEN you conduct independent interviews with employees at the bottom, to measure whether the new processes are understood at that level (bottom-up). If results aren't satisfactory, then add additional training or reengineer the processes.
Too often, it seems like this shit gets done at the VP PowerPoint level, and ground reality diverges without anyone noticing.
The map is not the world: interviews with a representative random sampling aren't hard.
No sane organization would ever implement this. If someone repeatedly makes mistakes, they're going to get fired even if the mistakes are unintentional. Anything else is going to cause more safety issues in the long-term as inadequate employees are allowed to proliferate.
There are always going to be some level of "inadequate" employees, and also perfectly adequate employees that sometimes make mistakes in any organization and if your organization requires that no employees ever make mistakes in order to operate safely, then you have serious problems.
The purpose of a statement like that is that you don't just have a post-mortem that is like: "Our company went off the internet because an employee had a typo in a host name. We fired the employee and the problem is solved." When in reality the problem is that you had a system that allowed a typo to go all the way into production.
https://sierrahotel.net/blogs/news/a-life-lesson
https://en.wikipedia.org/wiki/Bob_Hoover#Hoover_nozzle_and_H...
The situation is very different if the fuel cans are hard to distinguish vs if the tech is lazy and falsifying their checklist.
Underlying any safety culture is a one of integrity. No safety culture can tolerate a culture of apathy and indifference.
You apply risk mitigation and work verification to resolve safety issues.
Then you recursively repeat that to account for ineffective performance of the previous level of verification.
Ergo, end productivity per employee is directly proportional to integrity, as it allows you to relax that inefficient infinite (re-)verification.
This is why Boeing cant just solve their situation with more process checks. From the reporting, they are already drowning in redundant quality systems and complexity. What failed was the human elements.
Someone was gaming the system saying that the doors weren't "technically" removed because there was a shoelace (or whatever) holding them in place, Quality assurance was asleep at the wheel, and management was rewarding those behaviors.
Plenty of blame to go around.
There are no systems that are human proof, and what kind of human behavior is tolerated is a characteristic of the system.
In fact, there are humans that lie, cheat, are apathetic, and incompetent. Part of a good system is to not only mitigate, but actively weed these people out.
For example, if someone falsifies the inspection checklist for your plane, you dont just give them a PIP.
Why is it important to you?
I see lots of engineers and QA following a elaborate procedures with hundreds of checks, but not bothering to even read what they sign off on, so they can go golf all day.
People seem to think that you can engineer some process flow to prevent every error, but every process is garbage if the humans dont care or know what they are doing.
Every process is garbage is you dont hire workers with the right skills demanded by that process. In an effort to drive down costs, lots of companies try to make up for talent with process, with poor results, for both the companies and patients. you cant replace a brain surgeon with 2 plumbers and twice the instructions.
Similarly, I read some head of a leading engineering organization (I think a NASA head or maybe Admiral Rickover) who said, essentially, 'you can't replace ability with process'. All the process in the world, they said, will not substitute for highly able personnel.
But perhaps safety, not usually dependent on ability, is a different matter. Possibly, the problems you describe are a matter of leadership and management - which doesn't undermine your point; those also are things that can't, past a certain irriducible point, be replaced with process.
I wholeheartedly agree that leadership/management is a part of problem. My main objection is the "no bad employee" rhetoric. Sometime times the problem with management is that they aren't getting rid of bad employees. Rot can start anywhere in an organization, and the rest of the org really needs to push back, not just management.
It actually reminds me a lot of the culture/discipline problems with some Police departments in the US. It is hard to enforce and cultivate organizational culture top down. Most of it is maintained peer-peer.
> It is hard to enforce and cultivate organizational culture top down. Most of it is maintained peer-peer.
I think it's a combination. The leader has a large influence; they set the standards and the norms. At the same time, I agree with what you say about peers - perhaps peers spread and 'enforce' those norms. It may also depend on the size and age of the organization.
A lot of bad management is hand-waved as crappy employees (by management - shocking!)
If you are talking about someone who cant server coffee, the balance is clearly in favor of poor management over inadequate skills and trainability.
If you are talking about very specialized skills like aerospace engineering, I think the balance can move further in the other direction.
There is also the combination of the two, where in the interests of growth or cost savings, an organization has cut corners on the quality of talent hired.
[0]: https://www.controlinmotion.com/news/news-archive/a-little-h...
Imo it's a function of time, company and team culture, severity, and role guidelines.
If an employee makes a mistake but followed process, and no process change occured, that's just acknowledging the cost of doing business imo and would be a unbounded number of times so long as it's good faith from the employee
Not severity; that sort of thinking is actually part of low-safety cultures. A highly safe culture requires the insight that people don't behave differently based on outcome. In fact, most people can't assess the severity of their work (this is by design; for example someone with access to the full picture makes the decisions so that technicians don't have to). So they couldn't behave differently even if they did somehow make better decisions when it matters.
But, and I'll reiterate the point for emphasis, people make all their decisions using the same brain. It is like bugs; any code can be buggy. Code doesn't get less buggy because it is important code. It gets less buggy because it is tested, formally verified, battle scarred, well specified and doesn't change often.
Maybe impact relative to carelessness/aloof-ity?
I agree that an engineer/person will not behavior differently based on outcomes, but if they know in advance something can have a wide, destructive blast radius if some procedure is not followed, I feel there's a bit more culpability on the part of the engineer. Regardless I don't think I feel I have a sufficient grasp on this concept I'm trying to define so definitely agreed I shouldn't have included 'severity' in the function definition nor any alternative candidate
If they cant read the refueling checklist, they are a bad fit.
Ideally you have system controls to screen and weed these people out too.
Punishment culture assumes people naturally do bad, lazy things unless they are deterred by punishment and fear. Therefore we must punish mistakes.
That perspective has long been debunked. You don't see competent, skilled leaders using it. It turns out that generally people want to do well (just like you do), and they don't when they are scared / activated (in fight/flight/freeze mode), poorly trained, poorly supported, or poorly led. They excel when they feel safe and supported.
If you are the manager and the technician makes the same mistake the 2nd or 3rd time, you will find the problem the next morning in your bathroom mirror. :) At best, you have put them in a position to fail without the proper training or support. Leadership might also be an issue.
If your tech gets drunk every day and doesnt do their job, you need to cut them loose. This isn't a management problem.
Sometimes people end up in positions where they are not suited and will continue to fail. If you hired a plumber and you need a doctor, that isnt an on the job training, support, or leadership issue.
That is 100% a management problem.
> Sometimes people end up in positions
I wonder how they got in those positions? That sounds like a management problem too.
Some people are bad fits for positions. They might look good on paper, they might be trying something new, they might lie to get hired, they might change after starting, they might have been a risky hire, or any number of reasons.
As N approaches infinity, there's definitely a value of N at which we discover the root cause is the airman and have to move on from him. I don't think it's particularly interesting to try to identify a constant value for N because it's highly situational, and we know we have to do *just* and *reporting* as well, the reporting falls out when the just does.
All I am looking for is recognition that the content of N matters.
It is part of what I see as a broader phenomenon where people emphasize systems and ignore agents. In reality, agents shape systems and systems shape agents in continuous feedback.
Did the technician mess up (sometimes true), or were they doing their job in good faith - was it the system/protocol/organization that made the task mistake prone? Did someone else actually mess up but the situation made it look like it's the technician's fault? Does this technician do a task/service that is failure prone? Are there other technicians on other tasks that are far less failure prone? Here the former technician would seem poor, the latter, excellent, but it's a function of the task/role and not the person.
I've been "the technician" - I catch a lot of blame because people know I'm anti-blame culture, so I'd rather take the blame on myself that point my finger to the next guy in line. I'm also willing to take on high risk tasks for the greater good even if they suck and are blame prone / risky. I believe in team culture in this way. If the organization doesn't respect that belief and throws me under the bus, I leave - which is quite punishing for them since they remain completely unaware of a major internal problem. If an organization "sees me" and my philosophy, then together we get very very good at optimizing the system to minimize the likelihood of failure / mistakes.
That's a typical root cause, and is exactly what should come out of good post-mortems.
But human nature is human nature...
In fact, Just Culture in itself provides the justification for this. As the next line says "However, those who act recklessly or take deliberate and unjustifiable risks will still be subject to disciplinary action". A person who repeated makes mistakes is an unjustifiable risk.
But a policy that never provided for the possibility of termination (insufficient maximum severity) is also not desirable.
It's necessary if it's (necessary & efficient & justified); it's never desirable IMHO.
Doing severe things because they are justified is just acting out on a desire or drive - internal anger - but now we can 'justify' the target and feel ok about it. Lynch mobs think they are justified.
No one is designing a formal system that includes lunch mobs. But a formal system of repercussions for employee behavior that does not include firing is an incomplete system.
It’s not that firing itself is ever desirable, but rather that its inclusion in a disciplinary progression is desirable.
All indications here (from the NTSB prelim and the widely reported whistleblower account) are that during rework for a minor manufacturing discrepancy, the mechanics on the shop floor followed bad manufacturing planning / engineering instructions to-the-letter, then the ball was dropped in error handling when the engineering instructions did not match the airplane configuration, because Boeing was using two different systems of record for error handling that did not communicate with each other except though manual coordination.
That's not the fault of the front-line assembly worker not following a checklist.
I thought I was replying more to a parent comment addressing the inability to people go who repeatedly make mistakes, which is acceptable unless they are not following procedures.
And if someone keeps making new original mistakes, revealing vulnerabilities in your processes, I would say that it is a very valuable employee, a lucky pen-tester of sorts.
You described a fantasy world, in the real world everyone makes mistakes, and if the mistakes are punished, then there are no mistakes because no one reports them. That is until the mistake is so catastrophic, it cannot be covered up- that’s how you get Chernobyl or Boeing max
A mistake like "accidentally turning the machine off when it shouldn't be" is a fixable problem.
If someone has attitude like "fuck the checklist, I know better", it is not really a mistake, and that person should be rightfully fired or at least moved to a position where they cannot do any harm.
Your QC process has to be able to catch these things. It’s not like you can avoid any reliability in your process and just lean back and be like “don’t worry bro, it’ll fly, we hired all non-union workers”
I don't think you're charitably reading parent.
Sometimes, the employee is the problem. And sometimes, union contracts preclude changing processes (or make change unnecessarily burdensome) to improve outcomes.
Introducing another party into agreements doesn't come free.
I'm as pro-union as anyone around here... but it is another level of bureaucracy. And sometimes, protecting "our people" can include not letting go people who should be let go (e.g. decreasing headcount in assembly to hire more professional QA people).
That said, solutions do need to come from all levels, and I think unions do provide a necessary counter-pressure to management deciding only employees need to bear the brunt of change.
> where such fatally negligent parasitic workers can be easily shit canned
How charitable of a read does one need to give a comment that uses pejorative and hyperbolic language without providing any substantive refutation?
It takes self control, but walking past the problem is sometimes the more effective solution.
"Man, that asshole on the Internet disagreed with me. I'm going to write a detailed rebuttal of why they're the asshole."
vs.
"Hunh, I wonder why I occasionally get down voted and no one ever responds to me? And also, the responses I do get are a lot nicer in tone than mine..."
That being said, "operator error" as a reason for a problem is heavily discouraged (at least in official reports). Rather, it's typical to blame a human factors issue, commonly one of the "dirty dozen" [0]. I couldn't say exactly which ones are present at Boeing but frankly, from the public reports almost all of them sound applicable. With this in mind, process change is the only appropriate remedy, regardless of what the union thinks. I've never heard any union people get mad at something that improves their job though.
[0] https://skybrary.aero/articles/human-factors-dirty-dozen
In the 787 project they pushed outsourcing of design to their suppliers so they could reduce dependence on their own engineers. That was a disaster. They spun out Spirit Aerosystems to fragment their unionized manufacturing workers. The same Jack Welch influence via Harry Stonecipher sets the management tone even today. It's the wrong culture.
They need to move headquarters and production of their next plane back to the Pacific Northwest, and they need to develop the next plane not in the way that they developed the 787 which was which was designed to be a thumb in the eye of the union and their engineers. It's a cultural problem. Look at the word cultural throughout the FAA report. The culture that Boeing had doesn't work, and they need to change it root and branch.
Airlines are trying to get ahold of the planes built in Everett because the SC plant planes have quality control issues still, ten years in.
Pull the other one.
E.g., Boeing did not 1) classify MCAS correctly in their hazard analysis, and 2) even with their mischaracterized risk in the HA, they did not follow their own procedures to have redundant sensor readings mandatory for the equipment as classified. Those are designer decisions, not some labor-union issue. To my knowledge, those held responsible were relatively high level engineers, implying they were not being protected by the union for their decisions.
If it's a really critical role, the training will have realistic enough simulation for them to make countless mistakes before they leave the training environment. Then you can assess their level of risk safely.
Safety is a lot about trust, and there is more than one kind of trust. At a minimum: are you capable of doing this thing I need you to do? Will you do this thing I need you to do?
Goodhart's law also applies, as in the case of the edoor bolts, Spirit intentionally bypassed safety controls to meet performance metrics.
The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.
The Andon cord from the Toyota Production System wasn't present due to culture problems.
Same thing with impact scores in software reducing quality and customer value.
If you intentionally or through metrics incentivize cutting corners it will be the cost of quality and safety.
I am glad they called out the culture problem here. This is not something that is fixable under more controls, it requires cultural changes.
Challenger too. Multiple engineers warned them about the O-rings. They weren't just ignored, but were openly mocked by the NASA leadership. (https://allthatsinteresting.com/space-shuttle-challenger-dis...)
A decade later a senior engineer at NASA warned about a piece of foam striking Space Shuttle Columbia and requested they use existing military satellites to check for damage. She was ignored by NASA leadership, and following (coincidentally) a report by Boeing concluding nothing was wrong, another 7 people were killed by a piss-poor safety culture. (https://abcnews.go.com/Technology/story?id=97600&page=1)
Feeling safe is an illusion that governments try to maintain for their people. It’s one of their biggest jobs. But the illusion has dimensions and it’s hard to keep several going at once.
The fundamental goal is to distinguish between recoverable errors and those that are indicative of poor employee-role fit.
The point is to build a culture where you value teamwork and adjust and learn from failures.
This isn't an individual team problem, this is an organization problem.
It is impossible to hire infallible, all knowing employees.
But it is quite possible to enable communication and to learn from pas mistakes.
When you silence employees due to a fear of retribution bad things happen.
People need to feel safe with calling out the systemic problems that led to a failure. If that ends up being the wrong mixture of skills on a team or bad communication within a team that is different.
Everything in this report was a mistake, and not due to gross incompetence from a single person.
The E door bolts as an example was directly attributed to metrics that punished people if they didn't bypass review. The delivery timelines and defect rates were what management placed value on over quality and safety.
Consider the prisoner delema, which is resolved by communication, not choosing a better partner.
Part of maintaining quality culture is maintaining red lines around integrity.
Like I said above, not all errors are recoverable or honest mistakes.
I work in medicine and a classic example would be falsifying data. That should always be a red line, not a learning opportunity. You can add QA and systemic controls, but without out integrity, they are meaningless. I have seen places with a culture of indifference, where QA is checked out and doesn't do their job either.
Certainly nobody has ever thought about that before. In fact, there definitely isn't a second sentence in the definition of aviation's just culture that is being completely ignored in favour of weird devil's advocacy.
> 4) Just Culture- errors and unsafe acts will not be punished if the error was unintentional. However, those who act recklessly or take deliberate and unjustifiable risks will still be subject to disciplinary action.
Oh wait.
I simply agree that "that everyone has the right to work every job" is not a reasonable interpretation of them.
as stated above, a reasonable reader should understand:
> Not everything is a mistake, not every mistake is recoverable, and not all skills are trainable. The fundamental goal is to distinguish between recoverable errors and those that are indicative of poor employee-role fit.
This is why I called it weird devil's advocacy, because what exactly is the point of jumping to caution people about something they aren't doing?
Thats the parent in the thread we are posting in in. User Error-Logic replied, and I built upon their reply adding that:
>goal is to distinguish between recoverable errors and those that are indicative of poor employee-role fit.
You and others wanted to dive further.
Boeing has made numerous missteps in the last 15 years after being the world leader in airliners for around half a century. This only happens when knowledge about how to make a safe product is purposefully discarded and attempts to bring that knowledge back are intentionally ignored. In Boeing's case, it's due to desires for increased profits. They are unwilling to learn these lessons because it costs money that _may_ be there at quarter's end.
The opposite of 1-4 could be described as the "culture of lies, ignorance and fear". Fear is a good strategy for getting people working hard (if not always well) and lies make fear universal. Compartmentalizing information is needed to allow more and more functions to be subcontracted. If the company is extracting maximum value from it's assets this year, it has no incentive to report problems that will only appear in the future - by the time the future rolls around, the share holders have their and the shell of the remaining company can be tossed away. etc.
Also, another HN commentator mentioned how eliminating a culture of lies and retaliation is once it's in place. There's never a guarantee that those revealing a problem won't be punished once regulators turn their backs.
And 5 is only useful once 1-4 are in place. Otherwise, it's a culture of flexibly hiding your shit in different places.
Edit: This article was on HN a while back. https://qz.com/1776080/how-the-mcdonnell-douglas-boeing-merg... Key quote: These decisions, made by Boeing CEO Phil Condit, were made with a close eye on the company’s bottom line ahead of a hotly anticipated commercial-jet boom. An ambitious program of cost-cutting, outsourcing, and digitalization had already begun.
If this idea could be explored in depth, and more-or-less codified as received wisdom about market players, it would be a great contribution to management "science" and economics. My 0,02€.
It just confounds me (but explains a lot) that the manufacturer of the aircraft the airlines operate does not share a similar safety culture given that they are in a similar ecosystem (airlines report issues to the manufacturer and the FAA/NTSB all the time)
On top of that, Spirit Aerosystems was spun off so Boeing could demand higher production and lower prices, and fragment their assembly line workforce.
In this environment, when management has been hostile to their workers' unions, how are workers going to feel safe raising a red flag over "minor" production issues? You can't train for correct behavior when the incentives are so far out of alignment.
The five Key Elements of Safety Culture are:
1) Informed Culture- the organization collects and analyses relevant data, and actively disseminates safety information.
2) Reporting Culture- cultivating an atmosphere where people have confidence to report safety concerns without fear of blame. Employees must know that confidentiality will be maintained and that the information they submit will be acted upon, otherwise they will decide that there is no benefit in their reporting.
3) Learning Culture- an organization is able to learn from its mistakes and make changes. It will also ensure that people understand the SMS processes at a personal level.
4) Just Culture- errors and unsafe acts will not be punished if the error was unintentional. However, those who act recklessly or take deliberate and unjustifiable risks will still be subject to disciplinary action.
5) Flexible Culture- the organization and the people in it are capable of adapting effectively to changing demands.
Sources:
https://www.faa.gov/newsroom/Sec103_ExpertPanelReview_Report...
https://www.airsafety.aero/safety-information-and-reporting/...