News for hackers. Stuff that matters.
Re:Geoff Hinton Wants Computers to Think More Like Brains by chongli on Thursday January 03, 2019 @11:40AM (#18816416)
systemic failure involving large number of self-driving cars at the same time

This happens to human drivers every year. Snow, freezing rain, fog; these adverse weather conditions can lead to large numbers of collisions at the same time. Expecting self driving cars to never have a problem during rare environmental corner cases is to expect them to be vastly superior to human drivers all the time. If you're waiting for that day before you consent to self-driving cars sharing the road with you, you'll be waiting a long time.

Re:Geoff Hinton Wants Computers to Think More Like Brains by gambler on Thursday January 03, 2019 @12:43PM (#18817107)
>these adverse weather conditions can lead to large numbers of collisions at the same time.

Cars sliding out of control on a snowy road is not a systemic failure.

If you have an entire fleet of self-driving cars using the same (or similar) technology, the exact same condition could cause all of them to misbehave in the exact the same way. This could happen simultaneously with a large number of cars. Or could be something that propagates in a cascading manner. (A car breaks, turns and spins out of control. The next car detects that, breaks, turns and spins out of control, etc.)

If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?

People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving prioritizing safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it was designed and refined to be used by humans.

It's astounding how many of AI enthusiasts fail to see these issues and differences. Traffic system isn't a board game.

Re:Geoff Hinton Wants Computers to Think More Like Brains by johnwyles on Thursday January 03, 2019 @01:04PM (#18817362)
> If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?

A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them.

> People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans.

Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations.

> Traffic system isn't a board game.

I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject.

When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives.

We adapt.

Re:Geoff Hinton Wants Computers to Think More Like Brains by apta on Thursday January 03, 2019 @01:42PM (#18817813)
> Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging.

Some cars today already have IR sensors (night vision systems) installed in them, and include pedestrian detection monitors as well. Cars don't have to be full on self-driving to have a large increase in safety and accident prevention.

Re:Geoff Hinton Wants Computers to Think More Like Brains by YeGoblynQueenne on Thursday January 03, 2019 @08:32PM (#18821229)
>> A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them.

On the other hand, noone was ever run over by a mistyped URL.

Re:Geoff Hinton Wants Computers to Think More Like Brains by laythea on Thursday January 03, 2019 @01:39PM (#18817778)
How about: I like driving so don't take it away. or perhaps: Just because we can, doesn't mean we should.
Re:Geoff Hinton Wants Computers to Think More Like Brains by SiVal on Thursday January 03, 2019 @06:35PM (#18820472)
To guard against the risk of a bug/design flaw affecting all vehicles simultaneously, I would include an independent "dumb" override that was too dumb to drive but could stop all vehicles as soon as any of several TBD conditions occurred, regardless of what the primary "smart" system was telling the vehicle to do. It wouldn't be "smart" AI, but just something listening to both the AI's smart sensors and it's own, independent, simple sensors, and if ANY of them reported a listed condition, it would override the smart system and stop traffic.

Human intervention would be required to enable the vehicles to move again.

After some real-world experience, I would want to give the emergency stop system the ability to release the hold on one layer of vehicles at a time, allowing the smart AI to drive again but at a speed limited by the "dumb" system to nothing faster than human walking speed, enabling them to clear the area. If they cleared without further incident, the next layer would be released until the whole stoppage either cleared itself or was restopped because something was still wrong.

Re:Geoff Hinton Wants Computers to Think More Like Brains by darawk on Thursday January 03, 2019 @09:06PM (#18821392)
Self driving cars do not represent a correlated risk. The situation you describe is completely fanciful. There is no plausible circumstance in which a large number of cars would simultaneously crash.
Re:Geoff Hinton Wants Computers to Think More Like Brains by jcranmer on Thursday January 03, 2019 @11:00PM (#18821865)
Here's a link to a 100-car pileup: https://www.washingtonpost.com/news/dr-gridlock/wp/2018/02/0...

These kinds of crashes generally have the some root cause: everybody is traveling too fast for conditions, and when one thing goes wrong, the end result is that you get a chain reaction of everyone accreting into a single crash. If self-driving cars are capable of making the same misjudgements, then these kinds of crashes absolutely can happen.

Re:Geoff Hinton Wants Computers to Think More Like Brains by darawk on Thursday January 03, 2019 @11:36PM (#18821993)
Sure, but you can't criticize SDC's by saying that they have the same failure modes as humans. The comment I was referring to was suggesting that they'd have some sort of mass failure mode, in which some substantial percentage of all operating SDC's in an area would crash simultaneously. That is clearly not going to happen.
Re:Geoff Hinton Wants Computers to Think More Like Brains by neolefty on Thursday January 03, 2019 @09:10PM (#18821411)
A 0-day vulnerability in their OS, exploited ...
Re:Geoff Hinton Wants Computers to Think More Like Brains by m0zg on Thursday January 03, 2019 @12:32PM (#18817008)
>> Snow, freezing rain, fog;

Well, self-driving cars can currently handle none of the aforementioned conditions. That's why they are being tested where those conditions don't exist, and that's why we're about 20 years from widespread adoption. It rains and snows in much of the rest of the world.

Re:Geoff Hinton Wants Computers to Think More Like Brains by chongli on Thursday January 03, 2019 @12:40PM (#18817081)
It rains and snows in much of the rest of the world.

Yes, I live in the rest of the world and my family has been affected by fatal collisions brought on by adverse weather. I remain skeptical that self driving cars will be able to handle all of these things. I do think they'll be okay for mundane driving where the biggest risk is driver distractions, however.

I would like to see cars and maybe even roads made smarter in order to make driving safer. If all the cars on the road were part of a wireless network and they could signal to one another when they're braking or turning, then other vehicles could respond more quickly.

Heck, these systems could even be included in phones and pedestrian crossings so that cars could be signalled instantly when a pedestrian steps into the road in order to cross.

Re:Geoff Hinton Wants Computers to Think More Like Brains by Swenrekcah on Thursday January 03, 2019 @01:09PM (#18817432)
I used to think that this kind of system must be the future, but now I can only think of the greatly increased attack surface if my self-controlled car is not only taking directions from visual input and other sensors, but also all sorts of other unknown devices.
Re:Geoff Hinton Wants Computers to Think More Like Brains by chongli on Thursday January 03, 2019 @03:19PM (#18818900)
The system needs to be sane. The network should not be able to force your car to drive into a wall. Like any safety critical system, it should fail-safe. That means if the data it receives over the network would be unsafe to follow, given information from the other sensors, then it should discount that information as erroneous. This is something humans do all the time when, for example, reading a stranger's intentions.
Re:Geoff Hinton Wants Computers to Think More Like Brains by Swenrekcah on Thursday January 03, 2019 @06:03PM (#18820226)
But people are just too creative when it comes to exploiting systems. What if I force someone’s car to halt just under a bridge and then accidentally drop another car on top of it. Oops, a terrible ‘accident’! I realise this is a bit ridiculous but some other people are way more creative than I in this way.
Re:Geoff Hinton Wants Computers to Think More Like Brains by Cpoll on Thursday January 03, 2019 @10:33PM (#18821758)
What if I point a gun at someone and shoot them? As usual, the only thing that keeps you alive is that there are no motivated individuals that want to kill you. If there are, then they don't need you to drive a self-driving car.
Re:Geoff Hinton Wants Computers to Think More Like Brains by m0zg on Friday January 04, 2019 @08:00PM (#18829169)
You're ignoring the possibility of a mass-exploit, meaning the kind of cheap exploit that affects multiple vehicles at the same time, possibly killing a bunch of people, destroying infrastructure, or stopping the flow of traffic. Your typical adversarial "single pixel" change that makes your network believe unreasonable things, but in the real world. The existence of such exploits has already been demonstrated.
Re:Geoff Hinton Wants Computers to Think More Like Brains by m0zg on Thursday January 03, 2019 @03:43PM (#18819100)
I think there is a potential for them to exceed human performance in all driving conditions. Machines can perceive and react much faster than humans, and once the perceptual and control problems are solved, all of them will become uniformly "good" at driving in difficult conditions.

BUT (and there's always a "but"), the problem is exponentially hard, and it can't be solved with today's technology.

I agree with you that widespread adoption will require modification of roadways, and potentially also segregation of autonomous and human-controlled traffic streams.

Re:Geoff Hinton Wants Computers to Think More Like Brains by nradov on Thursday January 03, 2019 @02:12PM (#18818185)
Yes outside of a few limited areas (mostly controlled access highways) and benign weather conditions we're going to be waiting a long time for level 4+ self-driving cars.