Humans proved that for current level driving two not particularly high-resolution cameras are sufficient. Seems like pushing in this direction will remove this expensive component?
You get to use worse hardware, but you need several order magnitude better software.
I expect that, after the first wave of clumsy LIDARing self-driving cars, all the car companies (Google especially) will be collecting exactly training data from the cars' sensors to build exactly this kind of model. In fact, I wouldn't be surprised if that was what the Google car was really about, in the same way Google Voice is really about collecting speech training data.
The best part of this kind of training data is that it all comes pre-annotated with appropriate reinforcements: even if the image-recognition sensors aren't hooked up to anything, they're coupled to the input stream from the other car sensors and the driver's actions. So you would get training data like
- "saw [image of stopsign], other heuristically-programmed system decided car should stop, driver confirmed stop."
- "saw [image of kitten standing in the road], other heuristically-programmed system decided car should continue, driver overrode and stopped car."
Etc. Aggregating all these reports from many self-driving cars, you could build an excellent image-to-appropriate-reaction classifier.
In a driving situation, the worst-case scenario is everybody dies.
The human eye can instantly recognize the available driving paths, the motorcyclist ahead, and project where people will walk. Software would have to parse out where the open roads are, how far that motorcyclist is and whether he can clear the intersection before the car reaches it, and what that sign on the right-hand side is—using the same information, but it has to parse it first whereas we do that almost instantly. It's a totally different game.
Edit: a better solution would be to observe the behaviour of other drivers; if there is someone driving ahead of you, you can assume that the road between you and them is OK; if there's noone ahead of you, you need to drive slower and be more careful (that's how I drive at night). Once there's a critical mass of cars with cameras, cars could communicate road conditions automatically.
I would be frightened to trust the data coming from a random car in front of me. Inferring road condition from another car behavior sounds reasonable, using data supplied from it, not so much.
However, if the car three cars in front of you just broadcast "I'm doing an emergency stop right now", that's really valuable data. The human in your car won't know anything is wrong for at least a second. The human driver behind you would know about it before the human driver in front of you.
Until some asshat implements a button that sends this message for "lulz", though that would probably constitute a crime under existing law.
A computer-driven car, though, wouldn't (shouldn't) just immediately slam on the brakes because of that signal. It would tighten seat belts and start slowing down, but it also would want to avoid getting rammed by the car behind it. It can make very accurate estimates about its stopping distance and use all of it.
That's just my theory on it.
The simple answer is that we don't know how to do it with stereo vision alone yet. Getting range reliably is hard, and your brain uses lots of tricks to do it.
The second answer is that we need better-than-human performance if this is to take off. So using human-type sensing might not be good enough anyway.
Lots of researchers are pushing on vision-based driving, though.
Though, with many self-driving cars perhaps the sensors on each could share info, thereby mapping a larger (if not, endless) area.
My prediction: "crashdummy" will eclipse "heartbleed" and "shellshock"!
Tesla is talking about automatically putting their car into a garage. That's a good application; it's slow, and you can have sensors all around the car to avoid hitting anything. A more general system that can put a car into a big parking garage or lot is quite possible. Cooperating parking garages might have some additional bar-code markers and maybe a data link for open space info.
The whole airport car-rental thing could be done automatically, using slow-speed automatic driving to bring the car up to a pickup point at the terminal, just as the renter gets there. That may be one way this gets deployed. (I proposed that around 2003, but after 9/11, the idea of autonomous vehicles in an airport seemed politically hopeless.)
Those are some ways this might be deployed. Everyone has obsessed on automatic freeway driving since the 1950s, but that may not be the killer app.
It is one thing to anticipate threats, but when the road is under snow how do computers make that intuitive leap people can?
You need to be both looking at the road with cameras and profiling it with LIDAR. (Or terahertz radar, once that gets going.) It's not enough to just sense the car ahead. You need to be able to detect potholes, ice patches, junk on the highway, small animals, and similar problems. We could detect and avoid potholes back in 2005. Since we were doing off-road driving, that was a normal driving event.
The reason for a high-view LIDAR is that you want to see the pavement surface ahead from a reasonably useful angle and get a 3D profile of the road ahead. Google uses the Velodyne spinning-cone LIDAR scanner, which is a lot of LIDAR units built into one rotating mechanism. That's a research tool. There are other LIDAR devices more suited to mass production. Advanced Scientific Concepts has a nice eye-safe LIDAR which can operate in full sunlight. It costs about $100K, but that's because it's made by hand for DoD and space applications. The technology is all solid state, not inherently that expensive, and needs to be made into a volume product. (Somebody really needs to get on that. In 2004, I took a venture capitalist down to Santa Barbara to meet that crowd, but there was no mass market in sight back then. Now there is.)
You can only profile the road out to a limited distance, regardless of the sensor, because you're looking at the road from an oblique angle. Under good conditions, though, you can out-drive the range at which you can profile the road. That was Sebastian Thrun's contribution, and won the DARPA Grand Challenge. The idea is that if the LIDARs say the near road is good, and the cameras say the far road looks like the near road, you can assume the far road is like the near road and go fast. If the far road looks funny, you have to slow down and get a good look at the road profile with the LIDARs.
Automatic driving systems have to do all this. "Driver assistance" systems don't. Hence the "deadly valley".
That's just to deal with roads and static obstacles. Then comes dealing with traffic.