News for hackers. Stuff that matters.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by kqr on Tuesday January 10, 2023 @04:56AM (#34322551)
The actual report: http://www.fss.aero/accident-reports/dvdfiles/PT/2001-08-24-...

One of my pet peeves: the first 7 "causes and contributing factors" effectively blames the engine technicians for taking shortcuts. Nowhere does it say anything about the scheduling pressure from management that made the technicians consider shortcuts in the first place. If you have hiring and firing power and you tell someone they need to get the plane back in the air a few days later, they will do things to get the plane back in the air a few days later.

This is considered simply a "risk" by the report, and not a direct cause. As a consequence, the actual actions taken basically all belong to the category of "fix the engineers", "fix the flight crew", or "fix the plane". Not a single "fix the stressful maintenance environment".

Re:Crisis over The Atlantic: The near crash of Air Transat f... by Shank on Tuesday January 10, 2023 @05:54AM (#34322974)
It really underscores the importance of using a framework like HFACS [0] in a situation like this, because you end up exhausting all possible sources of problems and ensuring that you reach the correct root cause. It seems abundantly clear that the forcing-function was getting the aircraft back into service as fast as possible, not as safely as possible or as carefully as possible. A well-designed organization would have said that any unexpected contingencies during the maintenance process would have led to a full re-evaluation of the situation to understand it. The moment the engine didn't match their understanding, they should have scheduled a replacement plane or cancelled the upcoming flight, and reset expectations about how fast the plane could be back in-service.

[0]: https://en.wikipedia.org/wiki/Human_Factors_Analysis_and_Cla...

Re:Crisis over The Atlantic: The near crash of Air Transat f... by silvestrov on Tuesday January 10, 2023 @06:39AM (#34323306)
Also lack of access to documentation: the technician attempted to access the Rolls-Royce EIPC and the SB’s through the computer network, but was also denied access.

This should be a major point for a maintenance shop. Documentation should always be available, both online and in offline versions.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by hef19898 on Tuesday January 10, 2023 @08:19AM (#34324391)
Which it is, tech pubs are huge part of the after market activities of every aerospace manufacturer. That those are not up to date, or not available when or where needed, is a major finding during audits or investigations.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by gambiting on Tuesday January 10, 2023 @09:45AM (#34325577)
The article says the article was actually available on site through a different publication, but the mechanic wasn't aware of this(and yet did the right thing of contacting dedicated support, who didn't advise him correctly):

" The Rolls Royce SBs were also listed in the Trent Illustrated Parts Catalogue, accessible from any computer at the facility, but he was apparently unaware of this, so he instead switched to plan B and called the Air Transat Maintenance Control Center for help."

Re:Crisis over The Atlantic: The near crash of Air Transat f... by MilStdJunkie on Tuesday January 10, 2023 @10:40AM (#34326455)
So many problems involving tech pubs process here. I could almost classify this whole incident as a tech pubs problem, since the aircrew faults also involved the operational checklists and their planning/guidance.

TechPubs teams often have no control over the compliance (Optional, Mandatory, etc) of the SBs - that, at least in my experience, always came from way up high in the chain, sometimes from a VP who wasn't even tangentially involved in MRO. The required procedures - MX or ACRW - also are often coming from a costs concern, rather than criticality. The whole process of figuring out what's critical and what's not - that's a problem still being smacked into, every day.

It's not hard to make a Sanity Check for pubs vs ERP/PDM, but often people involved REALLY DON'T WANT TO SEE THAT. Because it's horrifying.

The commenter who mentioned HFACS is right on the money. Unless you implement something equivalent, you're gonna be chasing your own tail.

What a great article.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by dsfyu404ed on Tuesday January 10, 2023 @07:11AM (#34323594)
What shortcuts? Spec said no touchy between tubes. Tubes no touchy when aircraft left the hanger? Seems good to me.

These are the kinds of failures you get when you try to proceduralize everything with foolproof procedures, checklists, run-books, processes, etc. The maintenance people didn't even know they were making a judgement call.

Had the technicians sleeved the offending tubes with appropriate chafing gear and secured them to reduce movement as would be common place in more earthly situations instead of trying to get it "technically in spec" without thinking about the implications of what they were actually doing this problem may very well have been averted. But maintenance people in aviation aren't really ever exercising those creative problem solving skills since it's such a book driven workflow they're always dealing with so of course they missed the forest for the trees (not that they would have been allowed to implement a one-off fix like that).

Re:Crisis over The Atlantic: The near crash of Air Transat f... by bambax on Tuesday January 10, 2023 @07:30AM (#34323825)
> But maintenance people in aviation aren't really ever exercising those creative problem solving skills

This is the opposite of what happened. The engineers were confronted with a novel and unexpected problem, and instead of following procedure (which they lack access to) they applied "creative problem solving skills". From TFA:

> Although it was possible to install the post-SB fuel lines and hydraulic pump with a pre-SB hydraulic tube, the tube would rest against one of the fuel lines at a point where it rounded a 90-degree bend close to the pump. Aware that the plane could not be dispatched unless there was clearance between the tubes, the technicians torqued a nut on the end of the hydraulic tube until it rose approximately 0.635 mm off the face of the fuel line.

Torquing a nut so that the hydraulic tube isn't technically rubbing against the fuel line is pretty creative and seems to solve the problem. But it didn't take into account pressure and vibration while in flight, and almost caused the death of 306 people.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by iso1631 on Tuesday January 10, 2023 @08:38AM (#34324656)
I'm amazed they can measure the distance between two real non-fixed components down to a few micrometers (they wrote 0.635mm, implying somewhere between 630um and 640um)

If 0.635mm (an odd number itself which happens to be exactly 1/40th of an inch, or "25 thous" in American sizes) and they let it go that sounds really odd. That's literally the size of a grain of sand. If the concern is rubbing, sure that implies it will move during flight.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by bambax on Tuesday January 10, 2023 @08:47AM (#34324767)
0.635 mm is indeed an odd number. The thickness of a piece of 80gsm paper is 0.065 mm, so maybe what is meant is "about the thickness of ten sheets of paper"...?
Re:Crisis over The Atlantic: The near crash of Air Transat f... by jamincan on Tuesday January 10, 2023 @11:23AM (#34327231)
It's precisely 0.025 inches.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by intrasight on Tuesday January 10, 2023 @01:55PM (#34329640)
Who else thinks that if a tolerance of the thickness of a sheet of paper, in a mechanic/maintenance setting, could cause the death of over 300 people, that there's a striking lack of fault-tolerance in the design.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by plasticchris on Tuesday January 10, 2023 @08:48AM (#34324780)
It sounds to me like they stuck a machinists shim in there, effectively a bit of precisely measured metal.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by nick889996544 on Tuesday January 10, 2023 @10:00AM (#34325809)
Shim feeler gauges would be readily available in most shops checking wear on moving parts. 0.0025 inches (0.635mm) was included in the first high quality (gearwrench brand) set that I looked up online.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by mrguyorama on Tuesday January 10, 2023 @09:38AM (#34325471)
Plane maintenance involves a lot of precise and specially designed tools.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by mannykannot on Tuesday January 10, 2023 @10:03AM (#34325872)
I wonder if this nut, if not every nut on an ETOPS airliner (or in general), has a specific torque value. If adjusting the torque can open up a gap, presumably doing so could close one as well.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by lbriner on Tuesday January 10, 2023 @07:16AM (#34323638)
I don't think this is true at all

> These are the kinds of failures you get when you try to proceduralize everything with foolproof procedures

> Had the technicians

The report makes it clear that the problem was their lack of access to the relevant details, not that the process was defective. If they had seen the need to replace the hydraulic hose, they would have presumably changed it or flagged the problem to someone higher up.

> The maintenance people didn't even know they were making a judgement call.

It seems like they did. If you replace a part and something is touching, you are not trusted in aviation to try and make some space with a bit of fiddling for exactly this reason. They did make a judgment call. If they had relied on the process, again, they could have queried why the new pump was touching when the old one wasn't.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by mannykannot on Tuesday January 10, 2023 @08:35AM (#34324619)
There was a shortcut: "But if they had looked at the text of the SB, they would have realized that they missed a step: they were also supposed to replace the hydraulic tube which attached to the pump." With the old tube, the clearance was insufficient to avoid contact while in use.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by mrguyorama on Tuesday January 10, 2023 @09:41AM (#34325510)
"Follow the process" is because this is like the tenth example of engineers and maintenance personnel trying to ignore the instruction booklet and nearly getting hundreds of people killed.

"Read and follow the directions exactly" is a rule that is written in blood.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by mannykannot on Tuesday January 10, 2023 @10:12AM (#34325999)
> These are the kinds of failures you get when you try to proceduralize everything with foolproof procedures, checklists, run-books, processes, etc.

Aviation safety is utterly dependent on procedures, checklists, run-books and processes. The fact that things sometimes fall through the cracks is no argument against them.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by peteradio on Tuesday January 10, 2023 @08:45AM (#34324740)
Replace the use of the word "risk" with "gamble" and then it would paint the scene correctly.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by q1w2 on Tuesday January 10, 2023 @10:12AM (#34325992)
Semantics don't solve problems.
Re:Crisis over The Atlantic: The near crash of Air Transat f... by btrettel on Tuesday January 10, 2023 @08:59AM (#34324893)
That's a pet peeve of mine too. Here's a relevant HN comment I wrote about my experience as a patent examiner that applies more broadly:

https://news.ycombinator.com/item?id=31315234

> There are way too many jobs where management doesn't give the front-line worker an appropriate amount of time and/or resources, and when things inevitably go wrong, the front-line worker is blamed, and the problem is studied as if the only factor was the front-line worker.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by mannykannot on Tuesday January 10, 2023 @09:02AM (#34324926)
In this case, there is no evidence here of any ontoward pressure on the maintenance crew, and there is not even any evidence that anyone in the chain of command outside of those involved in the task had any inkling that this might develop into a schedule disruption.

On the contrary, the evidence strongly suggests that the maintenance crew did not feel they were under any special pressure: in their minds, they had found the correct solution, one which would allow them to complete the job on time, so there was no issue of flights being delayed or canceled.

You do not even have to work in the airline industry to see that unscheduled maintenance risks creating delays, and that is less than optimal for everyone involved - no-one has to be told it. If that knowledge creates too much stress and conflict of interest to be safe, commercial aviation would simply be untenable.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by dkarl on Tuesday January 10, 2023 @10:32AM (#34326326)
This reminds me of times in the first couple of years of my career, where me and some other junior people were terrified that we weren't going to get a project done on time. We would be wracking our brains for out-of-the-box ways to get it done, whispering to each other, having trouble sleeping, and then we would be amazed that the lead developer would simply go to our manager (while we were still desperately brainstorming) and tell him we wouldn't finish on time. And nothing awful would happen. The lead developer was fine. We were fine. The project finished late. Nobody got fired.

Now as a senior developer it's hard for me see why I was so terrified. Some managers yell and bluster. (Or, they used to, back in the day.) Some managers shame and insinuate. Some managers obliquely threaten. But they all, no matter how they express their authority, have to face up to the unpredictability of project planning. The question is not whether your manager is nice or mean. The question is, do they really think they're better off firing you and replacing you? Whether because they think they can get an upgrade or simply to pin the blame on you for something. After you've been around the block once or twice, you understand that a manager who is chill to your face is ultimately subject to the same motivations as a manager who uses stress as a weapon.

I wonder what it's like for airline mechanics? They are represented by unions. I would think that safety infractions would be the real potential career killers, but that's just me guessing.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by kqr on Tuesday January 10, 2023 @01:03PM (#34328872)
"It's not management's fault because engineers should expect to and put up with managers treating them harshly (yelling, shaming) when they are doing the right thing"?

Is this really the defense we're using?

Re:Crisis over The Atlantic: The near crash of Air Transat f... by dkarl on Wednesday January 11, 2023 @09:32AM (#34338861)
No, the point is that employment security depends on your manager's incentives, your employment protections, and your ability to get a new job if you lose one. This has very little to do with how your manager treats people.

Somebody who has been in the industry for a while and seen people fired and laid off has learned that someone's job is no more or less safe with a friendly, empathetic manager. (Your personal stress levels are another question, and an important one, but complying with a toxic manager doesn't decrease your stress levels, so it isn't leverage to make you do or not do anything besides change jobs.)

That's why I mentioned aviation mechanics being unionized, and I wondered if a worker's history of safety infractions follows them. Those would seem to be very important factors. A safety-critical worker should be able to say, "Look boss, you're telling me if I don't break this rule I could lose this job, but 1) the union has my back, and 2) if I do break this rule I might not get my next job."

That's important, because threatening in a loud and obviously abusive way is not the only way to make someone afraid for their job. You could even say that it's the amateur hour way to manipulate an employee. If management wants to scare people into skimping on safety, they can do it without doing anything you could point at specifically as "abusive" behavior, so you need protections that don't rely on identifying obvious misbehavior by management.

Re:Crisis over The Atlantic: The near crash of Air Transat f... by sidewndr46 on Tuesday January 10, 2023 @09:15AM (#34325104)
Management cannot be at fault because they are not responsible for safely flying the aircraft or safely maintaining the aircraft. That is the job of the employees, not management.

Now if management had for example knowingly procured inferior replacement parts or hired non-certified personnel to work on the aircraft, they would be at fault.