News for hackers. Stuff that matters.
Re:The Ethical Failures Behind the Boeing Disasters by jdkee on Thursday December 26, 2019 @05:58PM (#21887579)
I wonder if the MCAS software coders are held to any ethical safety standards by Boeing's engineering department or are they so siloed that it would be impossible.
Re:The Ethical Failures Behind the Boeing Disasters by skissane on Thursday December 26, 2019 @06:10PM (#21887665)
The programmer is told to implement a spec. They didn't write the spec. If the spec is flawed, do they even have the necessary domain knowledge and experience to identify those flaws? And, if they can identify them, and they speak up and nobody listens, what do they do next? Implement it anyway, or quit and let somebody else implement it instead? Especially if they don't even work for Boeing, but for some subcontractor.
Re:The Ethical Failures Behind the Boeing Disasters by ethbro on Thursday December 26, 2019 @06:15PM (#21887704)
The logic follows that if a malicious company deliberately subcontracts all their development, then they've completely indemnified themselves against all claims. While simultaneously providing no opportunity for subcontractors to bring noticed issues to their attention.

Which seems a pretty bad state of affairs to trust when building things like airplanes.

Re:The Ethical Failures Behind the Boeing Disasters by throwaheyy on Thursday December 26, 2019 @06:43PM (#21887852)
That is why high risk industries have regulators like the FAA instead of just relying on the fear of liability.

In terms of liability, Boeing can try push it to the subbies and the subbies can try and push it back to Boeing. Both are trying to bamboozle the non-technical lawyers.

The purpose of the FAA is to cut through that crap and enforce actual, effective change through sanctions or otherwise, and they didn’t do that. That’s what fell apart here.

Re:The Ethical Failures Behind the Boeing Disasters by to11mtm on Thursday December 26, 2019 @07:11PM (#21887988)
To wit:

In my state, Gambling is legalized. I remember my surprise when a friend who worked in the compliance side of the business actually knew what MD5 was (Back in 2007.) She wasn't a 'technical' person either.

She explained that they actually had to audit the slot machines to make sure that the code running on them had a hash that matched a codebase that had been audited and approved by the state regulatory body.

So, the practice for auditing code by a regulatory body is nothing new. If we do it for money, FFS can we do it when there are actual lives involved?

Re:The Ethical Failures Behind the Boeing Disasters by forgetfulusr on Friday December 27, 2019 @07:12AM (#21890887)
Someone weighed that the short-term money here was better than long-term. I think that quarterly earnings reports is the wrong optimization for some industries. If only the execs at Boeing could imagine each of their planes carrying suitcases of their Bonuses.
Re:The Ethical Failures Behind the Boeing Disasters by jdkee on Thursday December 26, 2019 @11:58PM (#21889321)
As for the FAA, they appear to be in the throes of regulatory capture on this issue.

https://www.economist.com/business/2019/03/23/regulatory-cap...

Re:The Ethical Failures Behind the Boeing Disasters by luckydata on Thursday December 26, 2019 @06:27PM (#21887779)
It is. that's why strong labor laws protect everyone, not just the workers, and why well run Unions are so fundamental to the sustainability of an economy.
Re:The Ethical Failures Behind the Boeing Disasters by _ph_ on Friday December 27, 2019 @04:42AM (#21890253)
Which is why it is so important to keep safety-critical software in-house. Because then the developers shouldn't be limited in knowledge about their prime domain, software engineering, but also about the domain the software is used for. So real aeronautic engineers need to be part of the software teams. People who would at least have a chance to detect fatal flaws in the specs they are supposed to implement, if they were not part of the spec creation from the beginning.
Re:The Ethical Failures Behind the Boeing Disasters by ethbro on Thursday December 26, 2019 @06:05PM (#21887639)
They were held to the standard of "Get it done, yesterday."

And will likely eventually be held to the standard of "How dare you let this happen?" as they're fired, to demonstrate how seriously Boeing takes safety.

Re:The Ethical Failures Behind the Boeing Disasters by jhayward on Thursday December 26, 2019 @06:11PM (#21887674)
I haven't seen any evidence that the coders were doing anything other than accurately implementing the system design that the control systems engineers dictated.

What ethical safety standards would you have them implement?

I assume that you would agree that the control system specification is the ethical responsibility of the control system design engineers.

Re:The Ethical Failures Behind the Boeing Disasters by salawat on Thursday December 26, 2019 @06:36PM (#21887814)
I don't implement anything I can't/don't understand.

I'm a generalist, so I make it my business to know a bit of everyone else's business. If I can't look at a spec without seeing issues down the chain that the spec makes no mention of, I end up feeling that it is my duty to make sure to raise the question until I am satisfied with the answer.

I don't always get the most satisfying answer, and I haven't had to put the career on the line by doing so yet; but I'm prepared to do so nevertheless.

I will not be part of the next THERAC-25/MAX fiasco. And if I've learned anything from this decade, it is that engineers as a whole may need to organize against those that would seek to have us do unethical work.

It wouldn't stop the practice, and God help me, I don't want the field locked behind accreditation/licensure...

However, I don't see any other defense or measure that would allow for putting the kebash on bad work. There has to be a price for bad corporate behavior in terms of ruthlessly pursuing performance that can only be met through wink wink nudge nudge style inducement to unethical behavior. At least, no way besides publically outting a company's dirty laundry. That really isn't satisfying though, because that requires a sacrifice of somebody's integrity every time, and no one wants to touch you after that.

I just can't converge to a satisfying middle-ground with the right incentives. Besides maybe anonymous whistleblowing to an appropriate watchdog agency. Even then though, issues are raised in that you are leaving the regulation up to people who feel insecure reporting something when they have everything to lose.

It is a frustrating issue to say the least.

Re:The Ethical Failures Behind the Boeing Disasters by throwaway890127 on Thursday December 26, 2019 @07:35PM (#21888105)
I've worked on systems in 3 categories:

1. Safety

2. Non-Safety

3. "Safety"

For #3 I mean it's "we realize that failure has bigger repercussions than a fail-whale, but we can't afford to do any of the ISO processes that have been proven to work." Sometimes I feel like my only job on those sorts of systems is to bang the "Normalization of deviance is not okay" drum in every meeting.

All failures need to go to the PM and get signed off on, otherwise the PM has a false sense of the actual reliability of the system. If the PM wants to get more budget for safety concerns, they should be able to hand a stack of 100s of pages of papers to whomever controls the purse strings and say "These are the failures in the last N days" If all they can say is "some of my engineers have expressed concerns" then 0 change will happen.

Re:The Ethical Failures Behind the Boeing Disasters by aidenn0 on Thursday December 26, 2019 @07:25PM (#21888057)
> I assume that you would agree that the control system specification is the ethical responsibility of the control system design engineers.

It is the primary responsibility of the control-system, but there is still also a responsibility with everyone who interacts with that spec to speak up if any flaws are noticed.

One of the big things that tight deadlines do is give tunnel vision to the engineers, so "just implement the spec" becomes the goal and the forest can be missed for the trees.

There were probably dozens of engineers that saw the MCAS specification as part of their duties; here's a few possibilities for what happened:

1. Nobody considered the case of improper MCAS engagement under normal flight conditions; this should clearly qualify the system for "Hazardous" classification under DO-178, which would require redundant AOA sensors.

2. Someone considered this case, but didn't speak up (was very junior, or it was way out their specialty).

3. Someone spoke-up, but was told by the person they spoke to disregarded it for the same reasons as #2, so it never made it to the control-system design team.

4. Someone spoke-up, it made it to the control-system design team, and business pressures caused the concern to not be investigated.

#4 would be significant ethical issues for the control-system design engineers, but I think it to be unlikely compared to the others.

#1 can be indirectly caused by time pressure. The certification process is supposed to slow things down, but there is some indication it did not sufficiently do so in this case.

#2 and #3 show ethical lapses outside the control-system design department, and are not just isolated to the individual in question, a safety culture needs to include cultural norms of speaking up about potential problems even when you think you are wrong.

Re:The Ethical Failures Behind the Boeing Disasters by MaxfordAndSons on Thursday December 26, 2019 @07:19PM (#21888024)
Seems unlikely given that they were likely contractors: https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-...