News for hackers. Stuff that matters.
Re:Researchers trick Tesla autopilot into driving into oppos... by dmitriid on Sunday March 31, 2019 @04:03PM (#19538001)
Adversarial attacks. Extrapolating from incomplete data. Consistent performance in any/most lightning conditions: night, dusk, low hanging sun over an icy road. Fog. Any combination of the above.

Humans while not perfect are capable if making split-second decisions on incomplete data under a surprising range if conditions based on the fact that our brains are an unmatched pattern-matching beast feeding off of our experience.

Re:Researchers trick Tesla autopilot into driving into oppos... by fattire on Sunday March 31, 2019 @05:22PM (#19538484)
Adversarial Attacks? Like removing a speed limit sign? Painting over lane markers? Dropping bricks off of overpasses? Throwing down a spike strip? Sugar in the gas tank?

Unless you're talking traditional computer security, which it doesn't seem like you are, these types of threats have not prohibited human drivers despite the fact that humans are very susceptible to "adversarial attacks" while driving too. Whether it's putting carefully crafted stickers over a stop sign to confuse a CNN or yanking it out of the ground to get a human driver killed.. you're talking about interfering with the operator of a moving vehicle... so what's the critical difference here?

Re:Researchers trick Tesla autopilot into driving into oppos... by dTal on Sunday March 31, 2019 @10:12PM (#19539862)
I think the difference is that software tends to be much more fragile - and more predictable - than humans. Paint fake lane markers, and an autopilot might drive full speed into a wall because it trusts them; the next 5 cars with autopilots will all do the same thing. An attacker can verify that an autopilot will do this ahead of time. A human on the other hand will be more likely to notice that things are amiss - they can pick up on contextual clues, like the fresh paint, and the fact they've driven that road hundreds of times before and instantly notice the change, and the pile of burning self-driving cars.
Re:Researchers trick Tesla autopilot into driving into oppos... by bnegreve on Monday April 01, 2019 @01:41AM (#19540812)
I'm not sure why hackers prefer to hack large scale computer systems rather than individual humans, but they do. So we have to protect neural networks against adversarial examples for the same reason we have to protect databases against sql injections.

> so what's the critical difference here?

If neural networks are deployed at scale in self driving cars, a single bug could trigger millions of accidents.

Printing an adversarial example on billboards would lead to crashes all around the country. Are we going to assume no one is going to try? (btw: real world adversarial examples are easy to craft [1]).

[1] https://arxiv.org/pdf/1707.07397.pdf

Re:Researchers trick Tesla autopilot into driving into oppos... by dmitriid on Monday April 01, 2019 @01:33AM (#19540767)
> Adversarial Attacks? Like

Like literally the article we’re commenting on. Image recognition systems in general are much more susceptible to errors in cases where humans wouldn’t even think twice.

And yes, “hey, a stop sign was here just yesterday” is also a situation for which humans are uniquely equipped, and computers aren’t.

Re:Researchers trick Tesla autopilot into driving into oppos... by sorokod on Sunday March 31, 2019 @06:06PM (#19538726)
Adversarial attacks with low friction for the attacker e.g malicious software updates.
Re:Researchers trick Tesla autopilot into driving into oppos... by fattire on Sunday March 31, 2019 @06:23PM (#19538818)
Do you mean OTA or just any attack because I think non-autonomous vehicles would have the same concerns... or really any equipment with embedded computers.
Re:Researchers trick Tesla autopilot into driving into oppos... by sorokod on Sunday March 31, 2019 @06:30PM (#19538849)
I mean any attack where the attacker is not required to move away from the keyboard and can corrupt multiple vehicles in one go.
Re:Researchers trick Tesla autopilot into driving into oppos... by fattire on Sunday March 31, 2019 @08:31PM (#19539428)
Could the keyboard be in a radio-equipped car?

https://jalopnik.com/its-scarily-easy-to-hack-a-traffic-ligh...

Re:Researchers trick Tesla autopilot into driving into oppos... by bsder on Sunday March 31, 2019 @04:44PM (#19538254)
Boy, are you an optimist.

Humans are LOUSY in almost all of those conditions as well as much less challenging ones--we engineer the road in an attempt to deal with human imperfections. The I-5/I-805 intersection in California used to have a very slight S-curve to it--there would be an accident at that point every single day. Signs. Warning markers. Police enforcement. NOTHING worked. They eventually just had to straighten the road.

Humans SUCK at driving.

Most humans have a time and landmark-based memory of a path and they follow that. Any deviation from that memory and boom accident.

This is the problem I have with the current crop of self-driving cars. They are solving the wrong problem. Driving is two intertwined tasks--long-term pathing, which is mostly spatial memorization, and immediate response, which is mostly station keeping with occasional excursions into real-time changes.

Once they solve station-keeping, the pathing will come almost immediately afterward.

Re:Researchers trick Tesla autopilot into driving into oppos... by dmitriid on Monday April 01, 2019 @01:36AM (#19540786)
Compared to the current generation of “self-driving cars” I’d say humans excel at driving.
Re:Researchers trick Tesla autopilot into driving into oppos... by hbarka on Monday April 01, 2019 @12:47AM (#19540580)
You make a succinct point. Can you elaborate on the difference between Station Keeping and Lane Keeping, what is generally available now as LKAS?
Re:Researchers trick Tesla autopilot into driving into oppos... by bsder on Monday April 01, 2019 @03:13AM (#19541191)
"Station Keeping" is maintaining your position relative to the other cars around around you--and it's what most people do when driving.

Ever notice how a bunch of stupid drivers playing with their phones tend to lock to the same speed and wind up abreast of one another? Ever notice how you feel compelled to start rolling forward at a light even when it is still red simply because the car next to you started moving? When in fog, you are paying attention to lane markers if you can see them, but you are also paying attention to what the tail lights ahead of you are doing.

All of that is "station keeping".

And it's normally extremely important to give it priority--generally even over external signals and markings (a green light is only relevant if the car in front of you moves). It's the kind of thing that prevents you from running into a barrier because everybody else is avoiding the barrier, too.

Of course, it's also what leads to 20 car pile ups, so it's not always good...