From Sterling Anderson, former director of Autopilot who oversaw the implementation of HW2:

>Perception is a game of statistics.Crudely speaking, if we have three independent modalities with epsilon miss-detection-rates and we combine them we can achieve an epsilon³ rate in perception. In practice, relatively orthogonal failure modes won’t achieve that level of benefit, however, an error every million miles can get boosted to an error every billion miles. It is extremely difficult to achieve this level of accuracy with a single modality alone.

>Different sensor modalities have different strengths and weaknesses; thus, incorporating multiple modalities drives orders of magnitude improvements in the reliability of the system. Cameras suffer from difficulty in low-light and high dynamic range scenarios; radars suffer from limited resolution and artifacts due to multi-path and doppler ambiguity; lidars “see” obscurants.

https://medium.com/aurora-blog/auroras-approach-to-developme...

He is basically just saying that the errors need to be independent, so you can get that with different capture mechanisms. I think that is wrong. For example, small, fast and poorly reflective will have similar detection performance for lidar and cameras. I think that the real problem is getting consistent statistical estimators and fusing them at a useful update rate. Has anyone seen the lidar from the uber crash? It would be interesting to see if a person could "tell" that the poor pedestrian wasn't an artifact quickly from lidar alone.

The radar, Lidar and cameras on the Uber that struck Elaine Herzberg all detected her, it wasn't a sensor failure that led to her death, it was a software failure, the autonomous OS tuned to not respond to positives below a certain threshold.

First, thanks for sharing this! Second - based on the language used I can’t tell if he is saying Tesla’s camera and Radar approach will work, or that it won’t. The only thing that I pulled out was that if we increase the types of sensors we get better results because they have different strengths and weakenesses.

Sterling does believe cameras may someday be good enough, but I'm personally of the belief that Lidar will get cheap before vision gets good, and as such there's no good reason to not have the extra redundancy.

I’ll take the “over” on that bet any day of the week. It might not be 20 years, but it’ll be a friggin’ while.

There are scenarios it’s possible to encounter on the road that humans don’t even necessarily know how to handle immediately. In proper, brutal, real-world testing, one very quickly enters the realm of problems that you’d need full-fledged general AI to reason through.

There is no upside to this approach. LIDAR is only getting cheaper, there is no reason not to use it.

Have you ever heard about that thing called fog?

Cars could communicate together though

I do not understand how anyone thinks this will work in any scenario that includes a malicious actor.

Most self-driving efforts do not consider malicious actors and instead aim for sufficient caution to avoid danger. Malicious actors can never be entirely accounted for, as a driver on a narrow road can turn into your path in moments and you can't avoid the impact.

I do not understand how anyone thinks it would be the only thing the car would use

As an existence proof, people drive through fog every day. We prove that it's possible to take binocular visual data and convert that into a series of control signals that will propel a car through fog. Sure sometimes accidents happen, but every day thousands upon thousands of people safely navigate fog in e.g. San Francisco or London. I see no reason machines shouldn't be able to perform at least as well one day.

Just as a point of reference here - I'm not taking sides in this debate, I vividly remember reading about the 1999 Toronto 87 car pile up.

Fog on a straight road caused an 87 car pile up after a trailer jackknifed. The point in the article I remember at the time was a mother recounting watching her 14-year old daughter burn to death. Mom got out, but the daughter had been pinned by her leg.

The quote from the time was ""Mama Sheila, please don't let me die. I'm only 14," Marceya McLamore begged."

I suppose my point is that neither human or machine are good enough yet. and not to treat this as an abstract academic topic, it's real people.

https://en.wikipedia.org/wiki/1999_Highway_401_crash https://windsorstar.com/news/15-years-after-401-crash-trauma... https://www.theglobeandmail.com/news/national/inquest-hears-...

I think olivewell's comment in this subthread is what dooms the idea of camera-only self driving for a really long time horizon: you basically need general AI for it.

I work in machine learning in a different area (healthcare) so my perspective may be incomplete, but what I see in ML/AI is that models are really good at memorizing, not so good at understanding. What I mean by that is that a human navigates the world by knowing what a car is, what fog is, what a person is, what a plastic bag is, all sort of things. Object detection can get much of the way there, whether it can get enough of the way there is an open question, but ok we'll grant that. Whether AI/ML can actually make the step from knowing what the object is to knowing how that object interacts with all the other objects in the world is another.

In other words identifying an object is just step one, understanding what that object means in context is the next absolutely required step, and it's way harder. We have a leg up as humans so we can rely on visual alone, but machines almost certainly will need other sensors that report additional information because they don't "get" context in the same way. And I'd say it's an open question whether even with those sensors, they'll be able to get there. I hope so, but the game is far from won.

Machines will be (and already are) held to a much higher standard than human drivers— there are a lot of reasons for this, ranging from the emotional to the pragmatic.

Just because thousands of people roll the dice every day driving in unsafe conditions does not mean that we should tolerate machines doing so.

One note is that perhaps people roll the dice because they want to feel some control and forward momentum. But if you're sitting in the car, being autonomously driven somewhere will you be so impatient?

I assume the car will give you an ETA, so you'll check the clock and go back to your book/social media. I'm okay with "slower, but smoother".

> I see no reason machines shouldn't be able to perform at least as well one day.

Assuming that day is infinitely far away, then you're right.

Why? What's magical about human beings that a sufficiently advanced sensor and sufficiently advanced AI can't do the same responsive task?

If there's magic in the human mind at all, it's the creative side of the mind. Driving isn't a creative act.