But when a supposedly 'all-seeing always watching' autopilot drives straight into a large stationary object in clear daylight, we have no understanding of how the situation occurred.
This I think has a couple of effects:
1) The apparent randomness makes the idea of these crashes a lot more scary -- psychologically we seem to have a greater aversion to danger we can't predict, and we can't tell ourselves the 'ah but that wouldn't happen to me' story.
2) Predictability of road incidents actually is a relevant piece of information. As a road user (including pedestrian), most of my actions are taken on the basis of what I am expecting to happen next, and my model for this is how humans drive (and walk). Automated drivers have different characteristics and failure modes, and that makes them an interaction problem for me.
Only when the vehicle computer detects a known object on the road that it knows should not be there it is applying brakes or trying to steer around.
I would feel safer if the algorithm would assume the negative case as default and only give the „green light“ once it determined that the road is free to drive on. In case of unknown (not yet supervised) road obstructions the worst needs to be assumed.
That’s where the ‚unexplainable‘ crashes are coming from. Something the size of an actual truck is obstructing the road. But couldn’t quite classify it because the truck has tipped over and is lying on the road sideways. Not yet learned by the algorithm. Can't be that bad, green light, no need to avoid or brake.
The problem with Tesla's "No LIDAR ever, cameras are good enough" approach is that it fails to detect emergency vehicles: they filter out stationary items out of radar signal as noise[1],and Tesla's ML models probably can't reliably identify oblique vehicles and semi trailers as obstacles.
1. Makes sense in isolation: frequent radar returns from roadside and overhead signs would be a pain to deal with
See the Tesla AI Day. I expect the new stuff to deal with this a lot better.
Trying to remember if the opposite of this is how human drivers are taught, or if this is implicit in how we move about the world. My initial gut reaction says yes and this is a great phrasing of something that was always bothering me about automated driving.
Perhaps we should model our autopilots after horses: refusal to move against anything unfamiliar, and biased towards going back home on familiar routes.
The answer was “the mile in front of you”
Additionally there was some statistic about the frequency of accidents within a very short distance of the drivers residence, which seemed to underscore the importance of being aware of just how much your brain filters out the “familiar” in contrast to a newly stimulating environment.
If I google it, I get like three pages of law firms.
I agree, but it will up the false alarm rate in a system without good depth perception for all objects. This is tough with cameras only. Reflective puddles are a problem; they're hard to range with vision only. Anything that doesn't range well, which is most very uniform surfaces, becomes a reason to slow down. As you get closer, the sensor data gets better and you can usually decide it's safe to proceed.
Off-road autonomous vehicles have to work that way, but on-road ones can be more optimistic.
Waymo takes a hard line on this, and their vehicles drive rather conservatively as a result. They do have false-alarm problems and slowdowns around trouble spots.
If the system gets faster over time, even better. But I cannot imagine huge adoption unless the system gets actually reliable. I am pretty much in favor of the Waymo approach.
To me, that blindness is simply unacceptable. If there is anything in the road, whether identified or not, it should automatically be flagged as a hazard. That flag should only be removed if it is detected to be moving in a way such that it will be somewhere else when you get there.
I have Subaru EyeSight. It has no problem seeing stationary objects. What's Tesla's problem?
Of course the vision system is supposed to compensate for this, and it performs poorly on objects it doesn’t see often, like emergency vehicles.
So, if they have the input data, why is it being ignored by autopilot?
It’s simply not possible to do depth estimation like this without priors. That’s one of the serious limitations of such systems - you have to train on every class of object you don’t want to hit.
It's one thing to have to deal with inexplicable behavior from other cars, but to have to deal with inexplicable behavior from your own car seems quite a bit more unnerving.
Honestly, I see this as a necessary transition pain towards fully automated vehicles. No matter how you slice it there's going to be periods where fully automated driving systems aren't quite there yet but are good enough 97% of the time that human drivers let their guard down. It's going to take some sacrifices to get to fully autonomous driving.
The good news is that even with these accidents self-driving features are a bazillion times safer than human drivers. It sure seems like the occasional vehicle collision into stationary objects is going to throw a great big wrench into self-driving safety statistics but it isn't even a rounding error compared to the sheer number of accidents caused by human drivers.
And yet, tens of thousands of drivers are working as unpaid beta testers for Tesla. Mind-boggling.
I don't see why these are inexplicable to humans. It's certainly no more difficult to explain than, say, a (non-adaptive) cruise control in a car from 2000 doing the same thing.
> Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor.
But that is arguably a sufficient explanation for these Tesla crashes as well. The driver being distracted or inattentive or unable to see clearly is a requirement for all of these Tesla crashes, as far as I know.
""" These events occur typically when a vehicle is partially in a lane and radar has to ignore a stationary object. This is pretty standard and inherent with TACC + radar.
The faster Tesla pushes the vision only stack to all cars after they’ve validated the data, the faster this topic becomes moot. Andrej Karpathy talks and shows examples of what that would do here. Minutes 23:00-28:00 https://youtu.be/a510m7s_SVI
Older examples from manuals of other TACC systems which use radar:
Volvo’s Pilot Assist regarding AEB/TACC.
According to Wired, Volvo’s Pilot Assist system is much the same. The vehicles’ manual explains that not only will the car fail to brake for a sudden stationary object, it may actually race toward it to regain its set speed:
“Pilot Assist will ignore the stationary vehicle and instead accelerate to the stored speed. The driver must then intervene and apply the brakes.”
Cadillac Super Cruise - Page 252
Stationary or Very Slow-Moving Objects
ACC may not detect and react to stopped or slow-moving vehicles ahead of you. For example, the system may not brake for a vehicle it has never detected moving. This can occur in stop-and-go traffic or when a vehicle suddenly appears due to a vehicle ahead changing lanes. Your vehicle may not stop and could cause a crash. Use caution when using ACC. Your complete attention is always required while driving and you should be ready to take action and apply the brakes.
BMW Driving Assistant Plus - Page 124
A warning may not be issued when approaching a stationary or very slow-moving obstacle. You must react yourself; otherwise, there is the danger of an accident occurring.
If a vehicle ahead of you unexpectedly moves into another lane from behind a stopped vehicle, you yourself must react, as the system does not react to stopped vehicles. """
The problem with radar on the ground is that most of what comes to a radar detector is reflections from a stationary world, with relative delays so small as to be undetectable. So the first step in processing is to filter out everything at the speed of that motionary world. All fixed objects therefore disappear, and you are left sorting out moving objects. Which means you now can't detect stationary objects at all.
Tesla has a different problem. They probably don't have depth perception. They therefore have to classify objects, and make educated guesses about where they are relative to the car. Unexpected kinds of objects, or objects in unexpected configurations, fail to be classified and therefore fail to be analyzed.
In principle, Tesla can succeed. After all we don't have binocular vision past 6 meters either. Tesla is improving.
But they haven't yet.
It's worth checking out for sure. Not worth the headline bandwidth and flamage budget being spent on it.
It's actually really annoying if you live in a rural area without clearly defined lanes, and large, stationary objects (tractors and whatnot) close to the road.
Additionally, the back up sensor is a tad over zealous also.
- Car switching in/out of my lane, I manually take over
- Tight curve in the freeway, manually take over
- Very frequently check the dashboard indicator that shows if the sensors "sees" the car front or not
- Anything unusual like construction, cones, car on shoulder, manually take over
- Anything that looks difficult like weird merging lanes, manually take over
- Any bad weather or condition like sun directly in front, manual drive
- Frequently adjusting max speed setting on ACC. It's safer to not be too much above the prevailing speeds. Otherwise, if ACC suddenly becomes blind, it can accelerate dangerously as it tries to reach max set speed.
- I don't trust lane keep much, it's mostly a backup for my own steering and making my arms less tired turning the wheel
The key thing is to recognize just how dumb this technology is. It's not smart, it's not AI. It's just a bit above the old cruise control. With that mindset it can be used safely.
If you use it that way it's fine. If you expect it to be as smart as a student driver it's not fine.
I'd rather at least get the pleasure of driving, rather than basically becoming the supervisor for my car. Quartlery performance reviews, checking on KPI's.
Autopilot and the like are absolutely not on my list of features I'm looking for when buying a new car. Crusie-control? Handy. AP, waste of (my) time.
I think that requires more numerate analysis than you're giving though. The data from the story is a sample size of 11 crashes over three years (I think). If that's really the size of the effect, then your "but [...]" clause seems very suspect.
There are almost two million of these cars on the roads now. It seems extremely likely that the number of accidents prevented by AP dwarfs this effect, so arguing against it even by implication as you do here seems likely to be doing more harm than good.
That doesn't mean it's not worth investigating what seems like an identifiable edge case in the AP obstacle detection. But that's a bug fix, not an argument about "Level 2 Autonomy" in general.
Actually, from talking with friends who are first responders, many times they will park fire trucks, etc so that they are blocking enough of the road to protect the first responders and the victims. The last thing you want is to have another car come and crash into first responders or victims of the initial accident. That is why they will deliberately park the truck at an angle to protect the people.
Larger departments or those dealing with busier freeways have even started re-purposing older engines with water ballasts and attenuators as 'blocker' engines.
https://www.sae.org/binaries/content/gallery/cm/articles/pre...
Good grief. This meme will not die. The car literally tells you to keep your hands on the wheel every time you engage autopilot, yells at you if you don't, will lock you out of the system as punishment if you don't comply, and if you really seem disabled will bring the car to a stop and turn the hazards on. It simply will not operate without an attentive driver, or at the very least one spending considerable energy at defeating the attention nags.
There are exactly zero Tesla drivers in the world who don't know these rules. Just stop with the nonsense. Please.
Tesla's marketing also knows that there are exactly zero drivers in the world who follow those rules, but that doesn't stop them from overselling the capabilities of what they ship.
1. concede to peer pressure and/or
2. doubt of the validity or seriousness of those warnings/lockouts
It only yells at you now because Tesla had to be forced to make it do so. Previously it'd let you go for a quarter of an hour before checking in on you.
Good grief yourself.
There are plenty of people who have been convinced that those safety features/warnings are “just there for lawyers” and have attached items to the wheel to defeat the safety lockouts in order to show off their “self driving car” to their friends.
"Tesla driver slept as car was going over 80 mph on Autopilot, Wisconsin officials say"
But no, just keep disregarding the clearly significant issues and mis-marketing, because progress, right?
https://www.forbes.com/sites/bradtempleton/2020/07/28/teslas...
It's clear what Tesla really has - a good lane follower and cruise control that slows down for cars ahead. That's a level 2 system. That's useful, but, despite all the hype about "full self driving", it seems that's all they've got.
"Full self driving" just adds some lane-changing assistance and hints from the nav system.