News for hackers. Stuff that matters.
Re:San Francisco cops pull over a Cruise driverless car for ... by carabiner on Monday April 11, 2022 @11:29AM (#30990149)
You didn't design the aircraft that you fly on, which are incredibly computerized. Also the drugs that your doctor prescribes. Etc.

You put your faith in myriad bureaucracies and trust networks that you have no hope of understanding. Driverless cars are just one more added to the list.

Re:San Francisco cops pull over a Cruise driverless car for ... by Veserv on Monday April 11, 2022 @01:18PM (#30991924)
The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed where as driverless cars have exactly zero software standards with respect to their safety or fitness for purpose. Comparing the processes in play is a outrageously fundamental category error; the systems have standards that are like 6 orders of magnitude apart.

As a example, just look at the 737 MAX, a airplane viewed as a literal deathtrap and a complete indictment of airplane software development, which had 2 crashes in 400,000 flights. At a average flight distance of ~500 miles, the 737 MAX, a airplane literally multiple orders of magnitude worse than any other and a complete failure of the approval process, only resulted in 1 fatality per 100 million person-miles. That is safer than the driving average of 1.1 fatalities per 100 million person-miles. What is viewed as by far the most atrocious outcome in the airplane sector, a system literally hundreds or thousands of times more dangerous than the average, is above average in the car sector; that is how far the gap is.

And this is ignoring driverless car software which is being developed to even lower standards than regular car software and which has no actual independent quality requirements. In contrast, far more lives are at risk with respect to driverless cars, so they should be developed to standards probably around 1000x better than airplane software which puts them around 0.0001% of the way there to a acceptable level of quality even if the software was already as reliable as regular car software.

Re:San Francisco cops pull over a Cruise driverless car for ... by jklinger410 on Monday April 11, 2022 @01:42PM (#30992230)
> The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed...

You mean the standards set by the FAA, which regularly defers to the private industry to regulate itself, due to complexity that it can't afford to keep up with? Those standards?

Re:San Francisco cops pull over a Cruise driverless car for ... by Veserv on Monday April 11, 2022 @02:21PM (#30992654)
Yes, those standards that produced a safety regime of 20 years of flights without software being even implicated in a single fatality. Literally trillions if not quadrillions of passenger-miles per software-related fatality. Those same standards which result in a system like the 737 MAX rightfully being excoriated for only being 10% safer than driving and resulted in successful demands for grounding of the entire fleet.

As you note, unfortunately the standards as employed in practice have recently regressed dramatically to allow a system as unsafe as the 737 MAX to fly. So it seems only prudent to go back to the historical, far more rigorous, process which did result in the amazing historical safety regime. And, since self driving cars are a even harder and more dangerous technology with far more downside risk, they should use a standard even more rigorous than the historical airplane standards which are still by far better, from a safety perspective, than any other consistently applied software process, such as cars, by multiple orders of magnitude.

Re:San Francisco cops pull over a Cruise driverless car for ... by jklinger410 on Monday April 11, 2022 @05:24PM (#30995290)
> Literally trillions if not quadrillions of passenger-miles per software-related fatality.

So these were all miles where people were unwilling testers as well. The difference being airplane auto-pilot is primarily focused on keeping the plane in the air, not dodging pedestrians. I think it does a lot of warnings for the pilots as well, but I'm not in the industry.

It's really comparing apples to oranges, isn't it?

> since self driving cars are a even harder and more dangerous technology with far more downside risk, they should use a standard even more rigorous than the historical airplane standards

So this is on The United States Department of Transportation to be far more involved in regulating the software that governs self driving. But the question is, was the FAA as relaxed in governing the aerospace industry 60 some odd years ago as USDT is now, and did they just get lucky that flying planes is actually easier than driving cars? Or is there actually some negligence involved?

Re:San Francisco cops pull over a Cruise driverless car for ... by mostertoaster on Monday April 11, 2022 @02:02PM (#30992492)
Yeah this is how most standards go. I use to work on medical devices, and the safety requirements for them were developed by my company and the other big players in the industry, and those became the FDA standards.

The FDA just wants there to be standards, and the companies have a vested interest in making sure those standards ensure safe medical devices.

Re:San Francisco cops pull over a Cruise driverless car for ... by olyjohn on Monday April 11, 2022 @02:01PM (#30992483)
Flying is still the safest way to travel. So I'm not sure what you're complaining about with the FAA.
Re:San Francisco cops pull over a Cruise driverless car for ... by nitwit005 on Monday April 11, 2022 @02:42PM (#30992897)
The crashes revealed the FAA wasn't doing the job people thought they were. We achieved safety in this case by grounding all of the planes after crashes, which is hardly the ideal method.
Re:San Francisco cops pull over a Cruise driverless car for ... by lcnPylGDnU4H9OF on Monday April 11, 2022 @02:12PM (#30992560)
I'm lost by your comment.

It seems that you're trying to suggest either 1) that the standards as mentioned necessarily aren't True Standards - as a blanket statement - because they're set by the industry or 2) that these standards aren't True Standards - specific to this particular discussion - because there is no reason to think they're necessarily more rigorous than the standards for driverless car software. If you're trying to suggest something else, please understand that it is unclear.

I'm all for a dissenting opinion, but when it is not presented in an understandable way the discussion becomes hard to follow.

(I don't mean to suggest that the argument is wrong when I call out the True Scotsman vibes; just that it is the argument I see.)

Re:San Francisco cops pull over a Cruise driverless car for ... by jklinger410 on Monday April 11, 2022 @03:45PM (#30993798)
What is the legal/regulatory difference between the aerospace industry regulating itself vs the auto industry regulating itself in regards to self driving? Besides the obvious that the aerospace industry has a head start.
Re:San Francisco cops pull over a Cruise driverless car for ... by lcnPylGDnU4H9OF on Monday April 11, 2022 @04:30PM (#30994508)
I believe I understand your point better.

From the person you're responding to:

> Yes, those standards that produced a safety regime of 20 years of flights without software being even implicated in a single fatality. ...

> As you note, unfortunately the standards as employed in practice have recently regressed dramatically to allow a system as unsafe as the 737 MAX to fly. ...

Your point, if I'm not mistaken, being that the FAA has clearly allowed for less stringent regulations in recent years. (Please correct me if I'm wrong. You can ask questions but it does not seem that they lead so immediately to the answers you expect.)

That being said, please consider that I am genuinely trying to understand you:

> I'm all for a dissenting opinion, but when it is not presented in an understandable way the discussion becomes hard to follow.

Re:San Francisco cops pull over a Cruise driverless car for ... by bsder on Monday April 11, 2022 @07:01PM (#30996260)
> The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed

Now they are. This regulations are a result of several tragedies culminating in a big disaster:

https://en.wikipedia.org/wiki/American_Airlines_Flight_191

Re:San Francisco cops pull over a Cruise driverless car for ... by ethanbond on Monday April 11, 2022 @11:41AM (#30990325)
Neither aircraft nor drug development norms look anything like the norms of software development. I actually find a substantial difference in my trust level between a traditional car company that's building self-driving capabilities (e.g. Mercedes) than a company that fashions itself as a software company that happens to build cars (e.g. Tesla) for exactly the reason of the norms and behaviors that each org would encourage.
Re:San Francisco cops pull over a Cruise driverless car for ... by woah on Monday April 11, 2022 @12:30PM (#30991142)
Didn't aircraft development norms lead to several planes plunging out of the sky with zero mechanical issues, due to software failure?
Re:San Francisco cops pull over a Cruise driverless car for ... by gouggoug on Monday April 11, 2022 @12:37PM (#30991273)
It's actually _failure to follow aircraft development norms_ that lead to the crash of multiple 737-Max
Re:San Francisco cops pull over a Cruise driverless car for ... by dogman144 on Monday April 11, 2022 @12:47PM (#30991442)
Generally agree but also knew people who worked on Boeing QA/SWE and the internal picture sounds dicy. I'm a bit disconcerted flying after learning about it.
Re:San Francisco cops pull over a Cruise driverless car for ... by skylanh on Monday April 11, 2022 @12:58PM (#30991642)
Boeing's issues were written on the wall about 2 decades ago when it moved it's hierarchy from an engineer-centric one to a Chicago accountant-centric one. That alone wouldn't have been an issue, but they had an immediate brain drain as half their HQ staff didn't follow the relocation.

Imagine a thousand employees, half your HQ staff, that leave. There are several possible reasons:

- they're older -> more corporate knowledge and knew the "magic recipes"

- skill based mobility -> they're so good they can go anywhere, and Chicago isn't Seattle

- etc

https://hbr.org/2001/10/inside-boeings-big-move

This was widely thought on ... what was I browsing back then? Reddit? as a fore-warner of bad stuff to come

The fore-warning came to fruition when the issues at various Boeing plants started developing over the next ten years (insider videos talking with an employee about being on drugs during crucial assembly work and shift mates being similarly high, "get it through your station" attitudes) and the Dreamliner issues that were hand-waved away (no training, 1-hour tablet self directed training, certification delays, and eventually crashes).

Re:San Francisco cops pull over a Cruise driverless car for ... by HeyLaughingBoy on Monday April 11, 2022 @04:17PM (#30994295)
> what was I browsing back then? Reddit?

I remember it being discussed on SlashDot back then.

Re:San Francisco cops pull over a Cruise driverless car for ... by foolinaround on Monday April 11, 2022 @03:21PM (#30993444)
generally, you dont want to know how the sausage is being made... its pretty much all industries, some worse than the others.
Re:San Francisco cops pull over a Cruise driverless car for ... by slg on Monday April 11, 2022 @12:01PM (#30990665)
Furthermore, at this stage the driverless cars are unlikely to be creating demand. These are replacing human drivers. Most of us have no hand in verifying the safety of other drivers on the road. I happen to believe it is bonkers that we generally test drivers once at the age of 16 and somehow that earns a person driving privileges for at least 50 years before being retested if not their entire life.
Re:San Francisco cops pull over a Cruise driverless car for ... by kylecordes on Monday April 11, 2022 @12:07PM (#30990776)
Imagine if cars were invented today. The bar would be much higher, probably halfway to how pilot licensing works.

(Of course this had some substantial downsides.)

Re:San Francisco cops pull over a Cruise driverless car for ... by njarboe on Monday April 11, 2022 @12:20PM (#30990964)
If cars were invented today, they would be too dangerous to be allowed to exist. Almost nothing has been invented since 1970 that can cause any harm. Safety first and all that.
Re:San Francisco cops pull over a Cruise driverless car for ... by notreallyserio on Monday April 11, 2022 @01:40PM (#30992209)
They'd have to be built with cow-catcher like devices to push all the horseshit out of their way while traveling down the road. It'd be a different kind of awful.
Re:San Francisco cops pull over a Cruise driverless car for ... by tomjakubowski on Monday April 11, 2022 @02:49PM (#30993003)
Personal computers cause all kinds of harm (in addition to the benefits).
Re:San Francisco cops pull over a Cruise driverless car for ... by njarboe on Monday April 11, 2022 @07:49PM (#30996741)
Not much physical harm and very little liability. I think computers and software was the one place people could make what they wanted and sell it without much worry of getting sued out of existence and that is why there was so much advancement. Software licenses are pretty amazing with the "as is" clause that seems to hold up.
Re:San Francisco cops pull over a Cruise driverless car for ... by monadmoproblems on Monday April 11, 2022 @12:04PM (#30990718)
Presumably, you choose whether to ride that airplane or take those drugs. You probably don't get to choose when someone else's driverless car runs you over.
Re:San Francisco cops pull over a Cruise driverless car for ... by helloworld11 on Monday April 11, 2022 @12:06PM (#30990762)
Nor do you get to choose when someone's airplane possibly crashes into your street from above.
Re:San Francisco cops pull over a Cruise driverless car for ... by gouggoug on Monday April 11, 2022 @12:43PM (#30991360)
There were 9 airplane crashes, worldwide, in 2021[1].

It's estimated 31,720 people died in car crashes, in the US, in 2021[2].

[1] https://en.wikipedia.org/wiki/Category:Aviation_accidents_an...

[2] https://www.transportation.gov/briefing-room/nhtsa-estimates...

Re:San Francisco cops pull over a Cruise driverless car for ... by helloworld11 on Monday April 11, 2022 @12:48PM (#30991465)
True, but you miss my point in referring to this. You still don't get to choose when someone else's mistake might harm you. Also, we as the public in general routinely trust planes with millions of risky flights despite not knowing anything about how well or safely they work, or how they work at all in detail. We accept that accidents sometimes happen nonetheless because that's how it goes with human technology that we've come to depend on. The important thing is that precautions are taken as much as reasonably possible. The same standard can be applied to driverless cars without getting fearful of them simply because of their newness.
Re:San Francisco cops pull over a Cruise driverless car for ... by gouggoug on Monday April 11, 2022 @12:55PM (#30991586)
But driverless cars do not go through the same requirements as aviation.

Before an airplane goes into circulation, it has gone through extremely rigorous testing. An airplane also always has a pilot.

The same can't be said for driverless cars. They don't go through extreme rigorous testing, they don't have a driver, and they're controlled by software that is most likely a far cry from aviation standards.

Re:San Francisco cops pull over a Cruise driverless car for ... by ratmice on Monday April 11, 2022 @11:47AM (#30990420)
Aircraft is a vastly different problem at a much smaller scale. Where did our ground traffic controllers go?
Re:San Francisco cops pull over a Cruise driverless car for ... by xyzzy21 on Monday April 11, 2022 @12:44PM (#30991390)
Except safety has only ever been improved for either aircraft or drugs by people dying from exactly this kind of failure.

Apparently there is ZERO learning or forethought about risks that can be anticipated or suggested from other technologies!

Re:San Francisco cops pull over a Cruise driverless car for ... by jjav on Wednesday April 13, 2022 @03:24AM (#31012033)
> Driverless cars are just one more added to the list.

As others have pointed out, aircraft certification (while it has its problems) is very rigorous. Driverless car certification, up to date, is basically nothing.

But the other aspect is that aircraft autopilots are solving a far easier problem. Developing a car autopilot for busy streets is many orders of magnitude more difficult than an airplane (or boat) autopilot.

Re:San Francisco cops pull over a Cruise driverless car for ... by akhmatova on Monday April 11, 2022 @12:59PM (#30991659)
Driverless cars are just one more added to the list

It's a qualitatively different change than any of the above (driven by a messianic view that AI is here and now and about to change everything), plus an instance of the

https://en.wikipedia.org/wiki/Boiling_frog

apologue.