If I had 360° night vision this wouldn't have happened. I can't trust my own two eyes. I wouldn't trust anyone's two eyes for that matter to drive a car safely. They weren't good enough last year and they won't be enough next year. Replicating humans will just replicate their weaknesses. I'll take all the crutches I can get.
By giving a car more than two eyes it has the potential to become a better driver than me because it can watch more. By adding more than just visible light cameras it can see things that I would never see. I want my next car to be safer, the big limit is not seat belts, brakes, crumple zones, airbags, and such - the big limit is the limited human in control. There is potential to make some of the above obsolete if the control system is enough better.
This isn't really the point, it's that humans are able to successfully build an accurate-enough model of the world to navigate terrain with only camera like vision -- and a really bad one at that.
There is, of course, no problem with giving cars more sensory abilities to make them safer but what they mean by a crutch is that the CS breakthrough that will make self-driving cars compete with humans will function about as well with two crappy cameras mounted on the drivers seat as it would with an array of high-precision sensors.
The idea that you can throw a cheap camera in a car and think you can achieve the same always seemed strange to me.
Unfortunately resolution of your eyes drops off quickly from the center of vision. Yes you can move your eyes to focus on different things, but so can cameras.
Sure the mental image you build is high resolution, but not that directly related to reality. A good example of this is the numerous optical illusions that depend on you looking at a point of an image, building a mental representation of that image, then finding a conflict whenever you move your eye.
Link that seems to be the source of the 576MP number: https://art-sheep.com/the-resolution-of-the-human-eye-is-576...
Not what I'd call a particularly scientific explanation.
True, but we can move our eyes around very quickly. Cameras in self-driving cars are usually fixed.
Also, human eyes have better dynamic range.
Let's not forget about human experience. It gives context to any visual perception, and can't be replicated by cars. We simply have access to a larger, more complex environment, and the environment fosters our intelligence. It's easy to overemphasise the brain to the detriment of environment, but in reality the environment is the main cause for the development of intelligence in the brain. The corollary is also true - a lower complexity environment will lead to lesser intelligence, no matter how powerful the brain is. Intelligence springs where agent needs meet with external limitations, those limitations guide its development.
By analogy, the environment is like the training set and the brain is like a deep neural net. A deep neural net with a poor training set is not going to be accurate. That's why DeepMind, OpenAI and others have focused so much on artificial environments (games) to train agents - they know that environments are the key to training advanced AI. A mere dataset is like a static environment where nothing new happens. Research is transitioning from datasets to simulated environments for the next step in AI. I'd even go as far as to track the evolution of AI by the evolution of environments and environment models.
As for the point about the training environment, a lot of what AI cars do is learn from simulations.
unless they wrote it down.
You can read all the books and notes about WW1 battles, you'll never come close to feel what they felt in the trenches.
Even if that's true, I (and almost anyone) can also drive cars in computer games at 1080p or less.
This is a spectacularly misleading claim. Humans have a very small field of high-definition vision, and a wide range of low definition vision, easily exceeded by almost any camera system. Where humans excel is in the heuristics of taking marginal information and teasing out meaning (not always the correct meaning), though neural networks approximate that system.
Tesla has something like 10 cameras running 1280x960 @ fps or similar. With so few data points (and no color) you can't identify similar vehicles. After all snow plows, school busses, ambulances, bikes, motorcycles, police, etc all have different behaviors. Some even signal cars with spot lights, police red/blue lights, brake lights, blinking high beams, etc. All would be invisible to lidar and contribute to lidar controlled cars acting less like human driven cars.
Biology got to be just good enough to make it to tomorrow. Photosynthesis is one of the oldest mechanisms around right? Quick google and at the moment that turns sunlight to energy tops out at 6%, now look at solar panels https://upload.wikimedia.org/wikipedia/commons/5/5d/PVeff%28...
Now, commenting on biology and electronics are topics vastly outside of my ability. I'm just trying to make the point that we can't be the best it can get, surely? We long left the time where industrial progress meaning "pick one things and do it well"
It's all about the data processing and the breakthrough that will enable human-ish self-driving cars will be there and not in more accurate sensors.
yep, resulting rich details simplify stereo matching a lot. Instead of complicated algorithm it allows our brain to just run simple displacement match in extremely parallel fashion natural to the brain.
>The idea that you can throw a cheap camera in a car and think you can achieve the same always seemed strange to me.
it isn't strange, it is just a bit early. Running 1MP stereo on a single core P4 in 2004 was just pitiful. These days a 20MP sensor costs under $30 and 16-32 cores is having much better time with it (using GPU is much cheaper and several times faster - it is just that i have these workstations around with good CPUs and no GPU to speak of. Tesla for example does run powerful GPU for their cameras.) So, scroll forward another 15 years, and we'll have nice stereo with something like 200MP sensors. I honestly don't see how lidar can do even just 4MP at minimally usable 30FPS in any foreseeable future (as probing at 150m is 1 microsecond per pixel).
However, the focal point with respect to the sensor is indeed miscalibrated in some units, and good units can become miscalibrated over time. There are some after-market correction devices available.
:)
At least as important, and something that comes up shockingly little as well (I don't see any responses to you yet on it here even!), is that overall humans stink at driving. Like you said, arguments begin with "because humans can do it" and need to be stopped right there, because that needs a lot of qualification. "Humans can do it"... with well over 1.25 million deaths a year worldwide [1] and another 20-50 million injured/disabled. Also these stats are actually worse then they look at first glance, because it's not a uniform age distribution. A lot of young people die on the roads vs other causes of death, which represents an even bigger loss in terms of human years.
Personalized arbitrary point to point mechanized transportation is so insanely valuable that it's well worth that kind of death/injury rate, or for that matter one vastly higher (as it was in previous decades before modern safety standards), but there is also no reason to use it as a benchmark for "acceptable" if the opportunity exists to do better. And in particular there is no reason to limit autonomous systems to human visual spectrum for information input. That doesn't cut it for a lot of adverse weather conditions, or even just for humans along the road or animals jumping across the road at night for that matter. A big part of the pitch shouldn't be to match humans, it should be to exceed them and by a sizable margin as well.
----
1: https://www.washingtonpost.com/transportation/2018/12/07/tra...
Some people claim that they won't make the same mistake twice, but how many times have you seen the same bug crop up after it was fixed in previous updates?
I would also posit that the existing statistics provide a "good enough" floor. If a million people currently die, the software will be good enough when it has the same level of fatalities. There is no incentive to make it better, at least not in our Capitalistic system. Investments would be better spent on differentiating widgets that dazzle the occupants, not protecting non-customers outside the vehicle.
This seems like a fallacious argument from incredulity. There is no reason to believe that humans are the ultimate existence for navigating around multi-ton land vehicles at 20-100+ miles an hour, and there is plenty of reason not to. For computers, there will be bugs and edge cases, but there are plenty of examples of high assurance low bug code in areas it matters. Additionally, any experience gained can be applied to the entire fleet at once. Computers have fundamentally better response times. And sensor input absolutely can better. A self-driving vehicle need not have any blind spots in 360°, and can "watch" all of that at once. No human will ever handle animals at night the way a vehicle with a FLIR could.
Meanwhile we know for sure that there are whole classes of issues that simply do not exist for computers. They do not get tired. They do not use drugs. They do not get distracted by the passenger or texting (humans are awful at multi-tasking). They do not "have a bad day". They do not get angry at that "jerk" in the other lane. They will know all the rules of the and not forget. Perhaps most of importantly of all, for all these things there will not be variability within a fleet. Sure, there are very good human drivers, but there are also endless awful ones, and every single person must be trained anew and go through a high risk phase while they get experience. Not with a computer. Consistency and attention to boring details and checklists, every single time, has proven a key driver of health & safety improvements across many industries. And even all that are just a few of the first order effects, there will be second order ones too like much less incentive to put off maintenance when the car can drive itself to the service center and another can take its place.
Sure there will no doubt be more obstacles along the way and it'll take a lot of time and effort. There may well be high profile screw ups, it's still humans doing the developing. But you've got a real steep hill in my opinion if you can look at all that and the casualty stats and still want to argue that computers won't ultimately be better.
>There is no incentive to make it better, at least not in our Capitalistic system
This is just utter bullshit though. Our capitalist system includes cost internalization in many cases, and cars are one of them. It's called insurance, and since you're required to carry it by law the fact that it didn't occur to you raises will questions in my mind about the basis of your opinions. Unlike many fields, in cars improvements in safety have a direct financial bottomline. Furthermore, self-driving will most likely enhance that, not detract it, because there the model of people giving up direct ownership and buying shares instead will become possible.
You also say lidar isn't a crutch because self-driving cars aren't ever really going to be able to go by image recognition and enough logic to know what to do - rather, they're going to go by the complete plotting of the traffic grid with lidar being there for object-avoidance. This is Google's model and they are furthest along - and set-up in Arizona where the unpredictable is kept to the minimum.
Indeed, and that's how autopilot driven buses, that were around for at least a decade, work. Except their movement trajectories are predetermined, and they are very slow, and stop at every obstacle imaginable.
I see that argument a lot, "well humans only have two eyes so of course two cameras would be enough".
Well monkeys have two eyes too but they wouldn't be driving on the autobahn anytime soon would they.
That argument is really used by people who have no idea how any of this works, and it really is frustrating.
Notably however, the radars employed by current Tesla cars lack the angular resolution to distinguish between stationary objects next to the road, and stationary objects in the middle of the road.
This is why Teslas keep running into stationary cars. The radar in them is not designed for this, and can't tell the difference between a fire truck in the lane and the barrier to the side of the lane.
But there are a lot of possibilities between zero autonomy and level 5 autonomy that might almost require lidar too in the short term.
Is there a better way to build a better autonomous car? We're at the initial stage of exciting people with an implementation that kind of works. We will now spend generations selling better and better cars with different, fewer, better, cheaper sensors and other parts.
I don't think Edison (and his employees) got a momentary flash of light from a lightbulb and then said, "alright, it's almost working, now let's see if we can do it without the fragile glass bulb."
Effective incandescent light bulbs were also around before Edison's. The main things Edison's team brought to the table were a better filament material, a better vacuum in the bulb, and supporting electrical infrastructure (generators, etc.)
In a perfect world, yes. But cost always comes into the equation. Companies like comma.ai and Nexar are getting their devices in to the hands of hundreds of thousands of people at this point. When you can get that level of training and autonomy from a simple camera based device it becomes a really compelling product, even though you're nowhere near full level 4 self driving.
Edit:
And neither is Tesla. Anybody operating either of these systems without paying attention to the road and being prepared to control the wheel at all times should have their drivers license confiscated.
See page 73 of the Model 3 owners manual: https://www.tesla.com/content/dam/tesla/Ownership/Own/Model%...
Warning: Autosteer is a hands-on feature.
You must keep your hands on the steering wheel at all times.
Warning: Autosteer is intended for use only on highways and limited-
access roads with a fully attentive driver. When using
Autosteer, hold the steering wheel and be mindful of road
conditions and surrounding traffic.*
Page 74 makes this even more clear:
Warning: Autosteer is not designed to, and will not, steer Model 3
around objects partially or completely in the driving
lane. Always watch the road in front of you and stay prepared
to take appropriate action. It is the driver's responsibility
to be in control of Model 3 at all times.Until it doesn't.
Failure to stop for a stationary gore point:
https://arstechnica.com/cars/2018/06/ntsb-autopilot-steered-...
https://abc7news.com/automotive/i-team-exclusive-tesla-crash...
Failure to stop for a stationary fire truck:
https://www.wired.com/story/tesla-autopilot-why-crash-radar/
Again, the Model 3 user manual is illuminating:
Warning: Traffic-Aware Cruise Control cannot detect all objects and,
especially in situations when you are driving over 50 mph
(80 km/h), may not brake/decelerate when a vehicle or object
is only partially in the driving lane or when a vehicle you
are following moves out of your driving path and a
stationary or slow-moving vehicle or object is in front of
you. Always pay attention to the road ahead and stay
prepared to take immediate corrective action. Depending on
Traffic-Aware Cruise Control to avoid a collision can result
in serious injury or death. In addition, Traffic-Aware
Cruise Control may react to vehicles or objects that either
do not exist or are not in the lane of travel, causing Model
3 to slowdown unnecessarily or inappropriately.
Warning: Several factors can affect the performance of Automatic
Emergency Braking, causing either no braking or inappropriate
or untimely braking, such as when a vehicle is partially in
the path of travel or there is road debris. It is the
driver’s responsibility to drive safely and remain in control
of the vehicle at all times. Never depend on Automatic
Emergency Braking to avoid or reduce the impact of a collision.
Warning: Autosteer is a hands-on feature. You must keep your hands on the steering wheel at all times.
Warning: Autosteer is intended for use only on highways and limited-access roads with a fully attentive driver. When using Autosteer, hold the steering wheel and be mindful of road conditions and surrounding traffic.
Warning: Autosteer is not designed to, and will not, steer Model 3 around objects partially or completely in the driving lane. Always watch the road in front of you and stay prepared to take appropriate action. It is the driver's responsibility to be in control of Model 3 at all times.
Warning: Traffic-Aware Cruise Control cannot detect all objects and, especially in situations when you are driving over 50 mph (80 km/h), may not brake/decelerate when a vehicle or object is only partially in the driving lane or when a vehicle you are following moves out of your driving path and a stationary or slow-moving vehicle or object is in front of you. Always pay attention to the road ahead and stay prepared to take immediate corrective action. Depending on Traffic-Aware Cruise Control to avoid a collision can result in serious injury or death. In addition, Traffic-Aware Cruise Control may react to vehicles or objects that either do not exist or are not in the lane of travel, causing Model 3 to slowdown unnecessarily or inappropriately.
Warning: Several factors can affect the performance of Automatic Emergency Braking, causing either no braking or inappropriate or untimely braking, such as when a vehicle is partially in the path of travel or there is road debris. It is the driver’s responsibility to drive safely and remain in control of the vehicle at all times. Never depend on Automatic Emergency Braking to avoid or reduce the impact of a collision.
It seems like the teams working on autonomous vehicles need all the crutches they can get, and it seems like a good idea to lean on superhuman sensors to make up for subhuman "cognition."
I do wish, though, that people would stop equating all radar with the adaptive cruise control style radar - imaging radars are a thing that exist, and can be competitive with lidar.