> Put another way - would giving humans superhuman vision significantly reduce the accident rate?

Yes? Incredibly?

If people had 360° instant 3D awareness of all objects, that would avoid so many accidents. No more blind spots, no more missing objects because you were looking in one spot instead of another. No more missing dark objects at night.

It would be a gigantic improvement in the rate of accidents.

Idk, many people are just bad at driving due to impatience.

They don’t leave enough space/time to react even if they did have enough awareness

Sure but that's not all people.

If you assume a constant rate of attention, but then massively increase what people are aware of during that attention, that's a massive increase in safety.

Safety is affected by lots of factors. Increasing any of them increases safety -- you don't need to increase all of them to get improvements.

... I don't know.

Many people are in accidents that are entirely not their fault, e.g. being rear-ended. You can't do anything about those who drive unsafely, distracted, under the influence, etc.

Assuming you have a "good" driver, there are still plenty of times they might be into accidents for reasons that aren't because of awareness. For example, road conditions, behavior of other drivers, and just normal, honest mistakes.

At least for me, my own sense aren't really the limiting factor in my safety. My eyes are good enough. Modern cars have the equivalent of bowling lane bumpers with auto-centering, pre-collision warning, and blind spot warning/indicators.

I don't understand what you're trying to say.

Your eyes aren't good enough, compared to if you had 360° LIDAR awareness, which is the comparison here.

You're listing all these other causes of accidents, which no one disputes. There's still a whole range of accidents caused by limitations in our spatial awareness because we can only ever be looking in one direction at a time.

You're talking about "normal, honest mistakes". That's what I'm talking about. There would be less of those if we had magic 360° LIDAR in our bodies. A lot less. Especially at night, but during the day too.

This isn't about blaming anybody. It's just the simple scientific fact that human senses are limited, and self-driving cars shouldn't limit themselves to human senses. Human senses aren't good enough for preventing all preventable accidents, no matter how "honest" drivers are.

The question was:

> Put another way - would giving humans superhuman vision significantly reduce the accident rate?

I'm saying that human vision is not the cause of most accidents. Most accidents are caused by distraction, incorrect decisions/errors of judgement, road conditions, etc.

And I'm saying a ton of those "incorrect decisions/errors of judgement" would no longer exist if we had superhuman LIDAR abilities.

You're using existing human vision as your baseline, which is not the right comparison here. The question is a baseline of superhuman abilities. So there's a ton of accidents we wouldn't get into if we had those superhuman abilities.

Basically everything involving a collision with an object we weren't aware of in time because it wasn't in our limited field of view and wasn't well-lit, that could have been avoided if we had been. Which is a decent proportion of accidents.

But the bigger, original point is that LIDAR is better than cameras, and better for avoiding accidents.

> No more blind spots, no more missing objects because you were looking in one spot instead of another. No more missing dark objects at night.

Of these, only dark objects at night is related to lidar vs vision. What percent of accidents is attributable to not seeing dark objects at night?

Limited field of view is solved by having a bunch of cameras, that is what is orthogonal to LiDAR vs vision.

I’m open to an argument here, but you have not provided any compelling rebuttal. You apparently feel like you have, though.

What are you even talking about?

You use some made-up idea like "superhuman vision" and then you're asserting that I haven't provided a "compelling rebuttal"?

You're not engaging in good-faith conversation here. If you want to understand the clear, obvious benefits of LIDAR over cameras, it's a Google search away. And it's not just about at night -- it's about weather, it's about greater accuracy, especially at greater distances, it's about more accurate velocity, and so forth. It's not rocket science to understand that earlier, more confident detection of a car or deer or child suddenly moving into the street is going to reduce collisions.

Maybe I need to explain better.

Obviously LiDAR improves the maximum achievable capabilities of a self driving system. I am not disputing that. I don’t think anyone would or could dispute that, it’s just trivially true.

The point I am trying to make is more nuanced than that.

Basically I’m thinking about whether this decision to ditch lidar is actually stupid, or if there could be a sensible explanation that is plausibly true. I am proposing what I believe is a plausibly true sensible explanation for the decision.

If you think about whether accidents would be reduced by having lidar, the answer is again obviously yes. But if you think about this as a business decision, it is not so simple. The question is not whether accidents would be reduced by having lidar. The question is whether accidents would be reduced enough to justify the added cost and complexity to the system.

So then, how do we figure that out? Well, we think about the causes of human car accidents ranked by percent of car accidents caused. What would be the items in the top 95% let’s say? Then, which of those items can be addressed exclusively by a lidar enabled system.

To isolate can-only-be-fixed-by-lidar problems, I imagine a thought experiment where we have a human with lidar for eyeballs. Which of the top k causes of accidents would be fixed by giving humans lidar eyeballs. Well, maybe not that many, actually. That is my point.

This could be argued to be immoral due to I guess making decisions about death based on business considerations. But it’s probably fairly easy to convince oneself as a Tesla executive that the car (M3 anyway) being affordable is critical to getting the safer-than-human driving tech adopted, so making the car more expensive actually fails to save more people than the lidar would save, etc.

Instant 360 vision doesn’t guarantee awareness. Most crashes happen in perfect conditions.

You seem to be missing my point.

Looking in one spot instead of another is included in what I’m calling “attention”. Of course paying attention to everything all the time would be and is a huge improvement. That is orthogonal to the type of vision tech being used. All approaches used in self driving systems today look everywhere all the time.

You were responding to a comment criticizing Musk for removing "radar/lidar".

So I'm assuming that the "superhuman vision" you described meant specifically if people had LIDAR, since that was the subject at hand.

LIDAR is superior to cameras because it much more accurately detects objects. It's an advantage all the time, and an especially huge advantage at night.

So LIDAR isn't orthogonal to anything. It's the entire point. If people had LIDAR, the accident rate would be significantly reduced.

You're arguing that LIDAR won't help cars, because by analogy it wouldn't help people if it was a native biological sense. But that's wrong.

The improved sensors improve awareness, improving situational awareness of human drivers would have a huge impact.

Humans have limited awareness bandwidth. I’m doubtful that improved “sensors” would change the fact that one can only take in and focus on very few things at a time. If anything, filtering down the input to the relevant features would consume more brain resources and possibly take longer.

Isn't radar/lidar less like super vision and more like spidey sense? I'd love to give human drivers an innate sense of exactly how far away things are and how fast they are closing.

> but vision quality is not really a major source of human accidents.

Never driven before?

I've driven before, and agree with the OP. Now what?

let's experiment. i'll reduce your vision quality, then you try driving

It's pretty obvious from the context of the thread that "vision quality" refers to LIDAR vs cameras, and that human vision is good enough, not that being legally blind doesn't somehow doesn't affect driving quality.

Human vision isn't good enough, though. Otherwise why not just drive the car yourself?

>Human vision isn't good enough, though.

Maybe if your standard is "scheduled commercial passenger flights" level of safety.

>Otherwise why not just drive the car yourself?

There's plenty of reasons why humans can be more dangerous outside of vision quality. For instance, being distracted, poor reaction times, or not being able to monitor all angles simultaneously.

> For instance, being distracted, poor reaction times, or not being able to monitor all angles simultaneously.

As a pitch for self driving, it's going to be a long time before I trust a computer to do the above better than I do. At the very least adding sensors I don't have access to will give me assurance the car won't drive into a wall with a road painted on it. I don't know how on earth you'd market self-driving as competent without being absurdly conservative about what functionality you claim to be able to deliver. Aggregate statistics about safety aren't going to make me feel emotionally stable when I am familiar with how jerky and skittish the driving is under visually confusing driving conditions.

Perhaps vision is sufficient, but it seems hopelessly optimistic to expect to be able to pitch it without some core improvement over human driving (aside from my ability to take the hand off the wheel while driving).

Edit: hilariously, there's already a video demonstrating this exact scenario: https://youtu.be/IQJL3htsDyQ

>At the very least adding sensors I don't have access to will give me assurance the car won't drive into a wall with a road painted on it.

This is as relevant as self driving cars not being able to detect anti-tank mines. If you want to intentionally cause harm, there are far easier ways than erecting a wall in the middle of a roadway and then painting a mural on it. If you're worried about it accidentally occurring, the fact that there's no incidents suggests it's at least unlikely enough to not worry about.

>Aggregate statistics about safety aren't going to make me feel emotionally stable when I am familiar with how jerky and skittish the driving is under visually confusing driving conditions.

Sounds like this is less about the tech used (ie. cameras vs lidar) and how "smooth" the car appears to behave.

But Tesla Vision is, currently, legally below minimum human vision requirements and has historically been sold despite being nearly legally blind.

Driving requirements in many states demand 20/40 vision in at least one eye [1]. 20/20 visual acuity is a arc-resolution of approximately 1 arc-minute [2] thus 20/40 vision is approximately a arc-resolution of 2 arc-minutes or 30 pixels per degree of field of view. Legally blind is usually cited as approximately 20/200 which is approximately 10 arc-minutes or 6 pixels per degree of field of view.

Tesla Vision HW3 contains 3 adjacent forward cameras at different focal lengths and Tesla Vision HW4 contains 2 adjacent forward cameras of different focal lengths and as such those cameras can not be used in conjunction to establish binocular vision [3]. As such, we should view each camera is a zero-redundancy single-sensor and is thus a "single-eye" case.

We observe that Tesla Vision HW3 has a 35 degree camera for 250m, 50 degree camera for 150m, and 120 degree camera for 60m [4]. The Tesla Vision HW4 has a 50 degree camera for 150m, and 120 degree camera for 60m [4]. A speed of 100 km/h corresponds to ~28 m/s as such the cameras trailing times of ~10s, ~6s, ~2s. Standard safe driving practices dictates a 2-3 second follow, so most maneuvers would be dictated by the 60m camera and predictive maneuvers would be dictated by the 150m camera.

We observe that the HW3 forward cameras have a horizontal resolution of 1280 pixels resulting in a arc-resolution of ~25.6 pixels per degree for the 150m camera and ~11 pixels per degree for the 60m camera, the camera used for the majority of actions. Both values are below minimum vision requirements for driving with most states with the wide angle view within a factor of two of being considered legally blind.

We observe that the HW4 forward cameras have a horizontal resolution of 2896 pixels resulting in a arc-resolution of ~58 pixels per degree for the 150m camera and ~24 pixels per degree for the 60m camera. The 60m camera, which should be the primary camera for most maneuvers, fails to meet minimum vision requirements in most states.

It is important to note that there are literally hundreds of thousands if not millions of HW3 vehicles on the road using sensors that fail to meet minimum vision requirements. Tesla determined that a product that fails to meet minimum vision requirements is fit for use and sold it for their own enrichment. The same company that convinced customers to purchase systems when they promised: "We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."[5] in 2016 when they were delivering HW2. Despite the systems being delivered clearly not reaching even minimum vision requirements and, in fact, being nearly legally blind.

[1] https://eyewiki.org/Driving_Restrictions_per_State

[2] https://en.wikipedia.org/wiki/Visual_acuity

[3] https://en.wikipedia.org/wiki/Tesla_Autopilot_hardware

[4] https://www.blogordie.com/2023/09/hw4-tesla-new-self-driving...

[5] https://web.archive.org/web/20240730071548/https://tesla.com...

>We observe that the HW3 forward cameras have a horizontal resolution of 1280 pixels resulting in a arc-resolution of ~25.6 pixels per degree for the 150m camera and ~11 pixels per degree for the 60m camera, the camera used for the majority of actions. Both values are below minimum vision requirements for driving with most states with the wide angle view within a factor of two of being considered legally blind.

This isn't as much as a slam dunk as you think it is. The fallacy is that assuming visual acuity requirements are chosen because they're required for safe maneuvering, when in reality they're likely chosen for other tasks, like reading signs. A Tesla doesn't have to do those things, so it can potentially get away with lower visual acuity. Moreover if you look at camera feeds from HW3/HW4 hardware, you'll see they're totally serviceable for discerning cars. It definitely doesn't feel like I'm driving "legally blind" or whatever.

https://youtu.be/Odu9O4MhfW0

That's called night and people drive in it just fine.

You are aware that more accidents happen per mile at night? If people drove at night just fine, this wouldn't be the case.

That just means we're successful 99.998% of the time instead of 99.999% of the time. It's not a particularly significant difference.

It's significant enough to directly prove that degraded vision leads to worse driving. This is very significant if we decide whether a driving system should only use vision, which can degrade.

Will limiting auto-pilots to human-level vision not increase their accident rate?

Yes?

Human eyes are in many ways far superior to reasonably priced vision sensors. This isn't giving humans superhuman vision, it's changing the tradeoffs human vision has (without changing the cognitive process that it coevolved with to begin, and which is the most important part of why we get into accidents).

There is no affordable vision system that's as good as human vision in key situations. LiDAR+vision is the only way to actually get superhuman vision. The issue isn't the choice of vision system, it's to choose vision itself, and besides the lesson from the human sensory system is to have sensors that go well with your processing system, which again would mean LiDAR.

If humans could integrate a LiDAR-like system where we could be warned of approaching objects from any angle and accurately gauge the speed and distance of multiple objects simultaneously, we would surely be better drivers.

seems like vision quality might be a major source of robot accidents