News for hackers. Stuff that matters.
Re:Toyota Research Institute head says full autonomous drivi... by gwbas1c on Monday August 28, 2017 @11:31AM (#15117195)
With software, it's always easy to make something appear to work. All the corner cases make something take 10-100x longer than the functional prototype.

This became painfully obvious with the article floating around where a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign. It's pretty clear that if a car is mistaking a red sign for a white sign, we have a very long time to go before the car can safely drive itself without a human observer to intervene when needed.

When it comes to automated driving, we need to keep our optimism in check and know that it can take decades before we have safe robotic taxis. They may become legal in countries with lower bars for safety before they are legal in the US.

The big problem, I think, will be when the drunks get into accidents because they will turn on autonomous driving, and don't get pulled over. Instead, they won't be sober enough to observe and intervene when the car screws up.

Re:Toyota Research Institute head says full autonomous drivi... by mikeash on Monday August 28, 2017 @11:49AM (#15117381)
The other day, I came within two feet of a crash because the other driver completely disregarded a red light.

It's pretty clear that if a human can't even pay attention to a light directly pointed at them, we have a very long time to go before humans can safely drive cars without a computer to intervene when needed.

Re:Toyota Research Institute head says full autonomous drivi... by frahs on Monday August 28, 2017 @12:08PM (#15117579)
One thing worth mentioning (even though I agree with the gist of your point) is that machines make the same mistakes consistently. This is pretty rare for a human. Imagine 10% of the cars on the road are autonomous and they all make this same mistake.
Re:Toyota Research Institute head says full autonomous drivi... by mikeash on Monday August 28, 2017 @12:17PM (#15117665)
There are a lot of road design features that come from the fact that humans do make certain mistakes consistently. For example, the US's love affair with four-way stops comes from drivers' inability to safely navigate uncontrolled intersections, even when there's no particular reason they couldn't. Stoplights usually have a delay between the red light for one direction and the green light for the next direction because driver's often run fresh reds, or get stuck in the intersection and need to clear out. There are tons of warning signs because drivers can't be trusted to notice something as simple as a sharp curve coming up.

Of course, the road system is designed for humans' foibles, not computers' foibles, and computers will have to deal with that. But the bar is low.

Computers aren't really as consistent as you say, either. Obviously, a deterministic machine will produce the same outputs for the same inputs. But when your inputs are camera data from the real world, you'll never get the same input twice. For example, my car sometimes misreads or fails to read speed limit signs, but it'll usually read the exact same sign perfectly fine the next time I go past.

Re:Toyota Research Institute head says full autonomous drivi... by bluGill on Monday August 28, 2017 @03:59PM (#15119573)
I wonder if in the future the law will say self driving cars can ignore all those signs when the car chooses to. Obviously there will be a lot of legalese about it, but most signs can be safely ignored already if you understand all the hazards - this is why people tend to just slow down at stop signs not come to a complete stop.

This is better for the environment, there is a lot of energy lost in a full stop for a stop sign that could be saved: less air pollution/CO2 to deal with.

Re:Toyota Research Institute head says full autonomous drivi... by euyyn on Monday August 28, 2017 @12:52PM (#15118053)
What models of car read speed limit signs?
Re:Toyota Research Institute head says full autonomous drivi... by mikeash on Monday August 28, 2017 @01:11PM (#15118225)
Teslas can (although the second revision of the Autopilot hardware doesn't do it yet).
Re:Toyota Research Institute head says full autonomous drivi... by andrewaylett on Monday August 28, 2017 @05:54PM (#15120405)
The rental Astra we had the other month did.
Re:Toyota Research Institute head says full autonomous drivi... by MarkMc on Monday August 28, 2017 @06:21PM (#15120598)
I think it's the opposite. Humans constantly make the same mistake like being drunk, tired, using a phone. With autonomous cars every single collision will be investigated - where the software is at fault a new improved version will be pushed to all cars, elimination the problem. This systematic improvement will mirror the improvement of airplane systems, but with a much faster time frame
Re:Toyota Research Institute head says full autonomous drivi... by ZeroGravitas on Monday August 28, 2017 @11:38AM (#15117260)
"This became painfully obvious with the article floating around where a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign. It's pretty clear that if a car is mistaking a red sign for a white sign, we have a very long time to go before the car can safely drive itself without a human observer to intervene when needed."

I think you misread that article.

Someone built their own toy implementation of an CV sign recognizer. Then, with full access to the code, they reverse engineered visual distortions that could seem innocuous to humans but trip up their model.

It's not a real world issue as far as I'm aware (though obviously signs can be misread or missed, but not invisibly vandalized)

Re:Toyota Research Institute head says full autonomous drivi... by jccooper on Monday August 28, 2017 @12:28PM (#15117800)
While they did use their own simple DNN classifier (because they couldn't find one), it was treated as a black box for purposes of the attack. A more robust classifier (probably using multiple methods) would be more resistant to attack, but the attack would still work, just the changes would be more obvious. At some point you hit "human-recognizable" but it's unclear where that point is.
Re:Toyota Research Institute head says full autonomous drivi... by averagewall on Monday August 28, 2017 @09:25PM (#15121492)
As long as you have to intentionally modify the sign to cause confusion, as opposed to doing random vandalism, I don't think this is a problem at all. It's like a person spray painting extra loops to change a 30 into an 80 or sticking a home-made sign over a real one so it's indistinguishable to most drivers. People aren't going to do that any more than they already do and it'll still be just as illegal.
Re:Toyota Research Institute head says full autonomous drivi... by Eridrus on Monday August 28, 2017 @11:41AM (#15117306)
Sure, some image classifiers can be fooled by some vandalism that requires a high degree of technical sophistication, but

a) Humans can be fooled by vandalising a stop sign too. The difference here is that the change is imperceptible to humans... but does this matter?

b) A lot of self-driving work leans heavily on maps which will know to expect a stop sign.

Re:Toyota Research Institute head says full autonomous drivi... by euyyn on Monday August 28, 2017 @12:54PM (#15118063)
It does matter if half the cars read a speed limit differently than the other half.
Re:Toyota Research Institute head says full autonomous drivi... by devmunchies on Monday August 28, 2017 @11:37AM (#15117247)
> a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign

That's what happens when the sign were made for humans. They should start putting QR codes under traffic signs, with the human sign as a fall back or as a second point of reference.

Re:Toyota Research Institute head says full autonomous drivi... by cmurf on Monday August 28, 2017 @12:51PM (#15118038)
QR codes are a goddamn eyesore. I'm voting against any city counselor who thinks they're a good idea. Find another way.
Re:Toyota Research Institute head says full autonomous drivi... by nayuki on Monday August 28, 2017 @11:42AM (#15117309)
How can you verify at a glance that the QR Code associated with a human-readable sign is correct?
Re:Toyota Research Institute head says full autonomous drivi... by devmunchies on Monday August 28, 2017 @02:37PM (#15118957)
I said this in my comment: "...with the human sign as a fall back or as a second point of reference."

You wouldn't remove human sign identification from the algorithm, you just supplement it with the small QR code under the sign.

Re:Toyota Research Institute head says full autonomous drivi... by Eridrus on Monday August 28, 2017 @11:47AM (#15117368)
Not that I think QR codes are a good idea, but why does this property matter?

People can be dicks to each other in myriad ways, I don't find these arguments that you could trick self-driving cars super compelling.