For tesla's, they have in the past struggled with stopped objects - so you have a tesla going straight into something at 70 mph on autopilot (highway dividers, stopped trucks, crossing trucks). They also suffer from phantom braking which is very disruptive, and the fix may have been to basically to disable emergency breaking in these areas which increases risks if something is stopped there. Lider, even lower rest, would really help in all of this plus pedestrian detection for emergency breaking there (see China videos of teslas crushing mocked-up walkers in videos there).
Google also is building absolutely nuts training data for wide stereo vision recognition if they wanted. They are driving these vehicles generating very hi def point clouds with synchronized video. In terms of a training data set for a vision based system, this is basically a dream situation. Ie, if google wanted to reduce lidar reliance, they may be well positioned for that as well.
Finally, what people want is level 5 autonomous driving. What trucking companies want is level 5. So they are skipping some of the earlier levels but getting to where I think people want this to go. Could a trucking company put a lot right near freeway and waymo get's the truck from lot to lot (1000's of miles) with then last mile hand driven? The savings would be huge there.
They are going to be in a great position when this all shakes out.
Maybe I have missed it, but has there been any substantial use of Waymo outside of the Phoenix area? There is a reason why both Uber and Google were targeting that area for testing. It is an area with lots of sun, little precipitation, lots of relatively new, big, and straight roads, no real wildlife that might jump out into the road, and that is notoriously hostile to pedestrians. Is demonstrating a self driving vehicle is safe there truly indicative that it will be safe in the back alleys of Boston's North End when there is a foot of snow on the ground?
It just works out better for the company since these drivers know their city and that is all they do. It keeps the company from having accident insurance claims.
I doubt there will ever be self driving Tractor Trailer type vehicles in any city. It requires some serious skill and intuition about your rig and roads, like where to take the truck wide to complete a right turn without taking down the street light or pedestrian waiting to cross would be a good example.
I'm actually convinced that maneuvering a large, articulated vehicle is the comparatively easy part.
These two videos convinced me AIs can be pretty good at it: https://www.youtube.com/watch?v=HR4MEh5-paA https://www.youtube.com/watch?v=BYPA4ajTQgk
AIs might even have an advantage over humans because they can more easily use cameras on all sides of the truck to maintain awareness of everything going on around them. The AI should be able to predict with good accuracy where each part of the truck will be as the motion progresses because it can do the math, whereas a human has to rely on experience.
To me, the real difficulty for AIs is in the higher reasoning, like how you deal with situations (lane closed, vehicle traveling the wrong way, etc.).
[1] - https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
https://venturebeat.com/2020/01/06/waymos-autonomous-cars-ha...
Either way, 20 million miles is once again a drop in the bucket compared to the number which would indicate these cars are as safe or safer than human drivers. For comparison, Tesla is likely around 4 billion miles with Autopilot. I still wouldn't be comfortable saying definitively that Autopilot is as safe or safer than humans.
If you have a very limited fleet of self driving cars then maybe it's actually ok if they are not as safe as human drivers as long as the trajectory is towards better safety. I'm sure the very first test car driving anywhere was less safe than a human driver. Each step will lay the foundation baseline for the next step. The Tesla 4 billion miles can't be compared with anything since those aren't really autonomous driving. It's like saying there's a trillion miles on cruise control.
Waymo has 20 million miles using multi-modal sensory data: cameras, LIDARs, etc. Because the sensory data is gathered in multiple modes, it can be cross-referenced and calibrated against each other even if the placement of individual components change (so long as not all components change at the same time), meaning that the data can be used across multiple models.
Tesla has 4 billion miles using a single mode of sensory data, but because it uses a single mode of input (visual), old data becomes mostly useless when they make any changes to the system, such as to the visual resolution of the cameras, or their positioning on the vehicles. This is one of the big reasons that they get so many regressions with Autopilot.
Also, despite having 4 billion miles of sensory data, that hasn't stopped Tesla from being the industry leader in self-driving fatalities. The rest of the industry combined has only a single fatality.
You are missing the point. Tesla is the industry leader in fatalities primarily because they have such a huge lead in the number of miles. Miles are not directly comparable to each other because each company has a different approach, but Tesla's fatality rate is 1 every 800 million miles. Waymo appears to be the leader among the rest of the industry with 20 million miles. How can you predict Waymo is safer than Tesla at this point? They may end up being safer, but it is way too early to say that right now.
And that problem gets worse with increasing data, not better, because the useless data will drown out the important data.
Tesla's own legal filings shows that Autopilot is 2-3x more likely to get into accidents that Teslas without Autopilot engaged. (The 2x for Autopilot disabled but other advanced driving features engaged; 3x for all advanced driving functions disabled.) https://www.forbes.com/sites/bradtempleton/2020/10/28/new-te...
Their marketing claims are just that: marketing.
Where is the source for this? It wasn't anywhere on the article that you linked.
It’s clearer by hours instead of by miles. If a normal car gets into an accident every 1 zillion hours, and I’ve driven my research car for 20 minutes and then had it parked for 100 zillion hours and it only had one accident accident, I don’t get to claim it’s 100x safer. (100x fewer accidents per hour!) It has to be apples to apples.
Second, you may be moving the goalposts. If the car can drive in Boston, but not in, say, French Guyana, would you still say it's not true self-driving?
You just described a quarter of the US.
If it works for the full spectrum of usage for normal people, I’ll count it, even if it’s not all normal people everywhere. If we get something that works everywhere in the US, but not the crowded streets in India, you’d count it, right?
In a safety critical system, I don't like hearing "we don't need x, we can get by with just y". I like redundant systems, especially when one may do better in certain conditions than the other.
Now I am not so sure anymore.
Looking at the beta UI in the car while it is driving, I am impressed with how well it detects everything. There are examples in dark and even areas with leafs covering parts of an unmarked road. It doesn't look at all like the detection system is struggling.
The only thing is, that things seem to "float" a bit, or cars and pedestrians disappears when going behind other cars and reappears again on the other side. The sides of the road also seems to float/shift a little bit from time to time.
I wonder if they are not using a temporal filter of some sort (like assume a car continues in the same trajectory when goes behind another object), or if the UI only displays an "unfiltered" value.
In any case, I still haven't seen examples where the cameras didn't pick up anything relevant/critical.
> Hey, Elon, a question on LIDAR. If LIDAR were totally free, would you want to use it in your cars near term? Would that tech significantly help Tesla on the training of your neural network for FSD?
> I mean totally free, well, I think probably — I think even if it was free, we wouldn't put it on.
https://www.fool.com/earnings/call-transcripts/2020/10/22/te...
If it turns out Tesla can drop in a cheap LIDAR system down then the line then aren't they the ones that made the right choice?
Waymo taxi looks incredible but how can they scale it.
1. LiDAR is expensive - this makes sense from his perspective because he has already declared all Tesla cars on the road are capable of “full” self driving with their current hardware. If he walks back on his, he would have to retrofit a million cars.
2. Humans can drive with just vision, so why can’t computers? This is a naive argument because human vision isn’t just vision, it’s linked to a ridiculous intelligence in the form of human brain. And computers are nowhere close to replicating it.
If you want to read more on LiDAR’s importance, read some of Brad Templeton’s writings.
https://www.templetons.com/brad/robocars/cameras-lasers.html
https://www.forbes.com/sites/bradtempleton/2019/05/06/elon-m...
So both sides of that sentence seem wrong.
EDIT: it's also worth noting that Tesla is accounting for a whole bunch of future revenue from FSD. If Tesla admits at some point that they can't make it with the current sensors that would have huge financial implications.
Also, I don't think Tesla is known for their AI expertise like Google. So it remains to be seen if they can pull it off.
You have to think at human level in corner cases, like seeing an object fly into view and judge whether someone is likely to be chasing it into the street, which depends on knowing what the object is and how it fits into people’s lives.
To cut corners and costs.
Point #2 is deeply flawed.
Source: I have blind friends.
Agreed this is very naive, it implies that humans are good drivers. Humans are awful drivers, and it sure doesn't help that [human] field of view and depth perception is very poor [compared to alternatives such as LIDAR].
Step 2: Theory of mind.
Step 3: ???
Step 4: Profit.
So yes the numbers are high but the item that a car using vision would have similar problems is a bit dishonest and LIDAR only provides one other means to determine where something is.
However self driving cars have one major advantage that LIDAR and cameras don't solve. They are not easily distracted. They are also far easier to be restricted to the ability of the vehicle they are managing adjusting for weather and more. Oh you can fool them for sure and there are always edge cases.
Also thanks for the link shares, but I don't think either actually disproves Tesla's approach.
My understanding of the Tesla approach is: In order to truly 'solve' self-driving (situations on the road that have never been seen before to drive safely - think unannounced construction, collision or road closures due to protests), you MUST solve 'vision' with a very, very complicated and well trained neural net (re: ridiculous intelligence in the form of a human brain as you state). In addition, the existing road infrastructure (re: signs) is all built around human vision - and so being able to identify and interpret all of that is a requirement.
I find their approach compelling as in this instance where you have cameras with a particular neural net (which they are constantly refining the learning model on) that are training across the millions of cars across the billions of miles across the thousands of various edge cases into a generalized solution. You also have a re-enforcement loop via the nature of a human driver which 'intervenes' through the drive, a necessary step in refining the model at scale. Note: I am not saying that Tesla's FSD will be coming to a street near you anytime soon. BUT, I haven't heard or understood a well articulated argument that says 'lidar is really the only practical solution'.
This also doesn't factor in that I've heard Lidar doesn't work well in any kind of precipitation (light being refracted away from the sensor). Also, full self driving doesn't mean it can drive in situations that humans WOULDN'T be comfortable with (i.e. snowy blizzard or thick fog) so in either solution shouldn't be factored in as a part of the required solution set.
And finally, the practical threshold on a FSD system that would pass regulatory approval is evidence/data that it is materially safer than a 'typical' human driver. It seems the 'throw millions of cars and billions/trillions of miles at it' with a refined tagging system that approaches the narrow vision solution for driving forward in 'driveable space' seems to be most likely to reach a solution first.
Yeah this is dubious. You need to solve situational awareness. Vision is one way of doing this. Lidar is another, and lidar avoids many of the drawbacks of vision (having to do accurate world modeling based on cameras).
Tesla doesn't (and would be stupid to) feed camera data directly into a neural network. They feed multiple cameras into a complex system that involves both classical object positioning and neural networks to build a model of their surroundings. Then a downstream system consumes that model and makes decisions.
Its not a single end to end black box. Such an approach would be computationally infeasible, not to mention over-parameterized to all hell. No one does this, not Waymo and not Tesla.
While cameras are good at certain tasks (like detecting traffic lights), they are not good at all tasks, and using more specialized hardware for object detection and world modeling means that lidar based systems are strictly better. They have more information than camera based ones.
Tesla is betting on, somehow, making some breakthrough in computer vision that no one else can replicate, and further that lidar can't do what cameras can.
Your argument appears to be that since Tesla has more data, they'll achieve some eventual success, but the point is that they'd achieve more success faster with lidar, and everyone in CV seems to agree that we'd need pretty fundamental improvements in CV (and perhaps in cameras) before you get the same performance out of CV that you get out of lidar. That means that Tesla's betting on a less accurate world model being good enough. Maybe they're right, but so far we have some evidence to suggest that cameras alone have some pretty fatal shortcomings, and no evidence that Tesla has solved them.
The models they use are incredibly opaque, complex, and work based on statistical inferences that are built into the data. Making fundamental changes like assumptions about where that data came from is not trivial. Even more so given that we're talking about safety-critical applications.
This has been demonstrated many times by Teslas crashing into stationary or large obstacle in front of the cars that would have been immediately apparent to a human driver or a car with LIDAR.
That's why everyone else in the industry use both types of sensors.
People are actually terrible at driving, even when paying attention.
I really hope that the future of self driving reduces the road toll to a teeny fraction of its current level.
1. They are 10s of billions deep in a hole and make no profit (or revenue) and will not for multiple years. To ever pay back that investment they basically need to become Uber. And to get there, they need many more 10s of billions of capx. Uber is not even making a profit yet.
2. Waymo approach is the classic 'solve the easiest problem first' mistake. They started and have driven most of their miles, in literally the easiest place to do AV in the world. They have created incredibly complex high resolution maps on one small and simple region in Arizona where the streets are wide, the weather is always nice, and the general upkeep of the infrastructure is done very well.
3. Reliance on hd mapping for localization is fragile approach, and it essentially requires you to continuously map the whole global road system to have a real generalized driving solution. When will they have cars that can just drive on roads they have never seen before, something that human drivers often do. Seems to me this is the wrong approach to take.
4. Waymo still needs highly specialized vehicles, even with lidar becoming cheaper, their whole stack is still a very high cost low volume stack. How they will compete against an Uber driver in a cheap mass produced car anytime soon is totally dubious to me.
Tesla clearly does not want Lidar. Elon Musk even said he wouldn't use Lidar if it was free. What they are actually doing is using next generation radar and likely next generation sonar as well. See: https://electrek.co/2020/10/22/tesla-4d-radar-twice-range-se...
I don't believe that. He ultimately has to be anti-LiDAR because putting LiDAR on every Tesla is simply not feasible nor compatible with his plan to mass gather training data, so he has to justify it somehow.
What you're missing is that it's all moot if your self-driving car does not have a 0% failure rate, or close enough to 0% that it rounds down to 0 accidents. As soon as you have real accidents, you are done. Look at Uber. The one death completely ended their self driving unit.
So yes, it may seem cool to just go with the flow and rush things, but I get the feeling that this will be like the tortoise and the hare. Eventually LiDAR will be cheap, and Tesla will be stuck with years of useless training data.
To me, the revolution that actually mattered the most was the cell phone supply chain revolution (if you can call it that). That is what allowed these companies to equip their vehicles with cheap cameras, radar, and eventually LiDAR.
> I don't believe that. He ultimately has to be anti-LiDAR because putting LiDAR on every Tesla is simply not feasible nor compatible with his plan to mass gather training data, so he has to justify it somehow.
Why is using Lidar not compatible with mass gather training data?
Also he recently said they wouldn't use Lidar even if it's free. He didn't leave the door open for when the price goes down sufficiently. He could be wrong obviously, but I don't think he is in denial.
https://www.tesla.com/support/full-self-driving-computer
I suppose they wouldn't have any problem asking for Lidar installation if that were to be required?
2. Solve the easiest problems first, as opposed to what? Even the easy problems are hard to solve, trying to solve everything at once is going to be floundering around with nothing to show. Is an automated vehicle that can only do the easy parts super useful? Not really, but it's somewhat useful. In a model with human controls, this is tractable, assuming there's a safe handover --- either directly with appropriate advance notification and acceptance, or by parking and switching; without a safe handover or human controls, it's pretty limiting, but some areas would have most of the year coverage, and some areas have tourist populations that overlap with fair weather, so it's something.
3. I don't think this is a big problem, if the vehicles are driving the routes, they can update the mapping. The road map is finite, and most places don't change that often. Again, safe handoff is critical, though.
4. Yep, it's expensive. But mass produced, less expensive lidar is plausible. Given this thing is still several years from mass deployment, constraining to today's inexpensive tech is too limiting. Figure out if you can do it, and figure out how you can do it with unlimited budget, and then figure out how to make a working system with less.
P.S. I can't believe I'm defending Waymo here. Usually I'm the staunch negative against automous vehicles. On the other hand, Waymo's serious errors have been more amusing than dangerous (let's try to merge into a bus at low speed, cause it'll certainly move out of out way)
If your mental model of how the world works is "Elon Musk always tells the truth" then this is compelling evidence, but if you have a different mental model I don't know why you would take it seriously.
The important point is that he is CEO and its well within his power to develop his own lidar system, or sign a contract for lidar. Or to do research on the topic. Yet he has not done that or shown any interest in that.
At some point you have to ask yourself, is somebody engadging in 4D chess over 10 years or is it simply his opinion.
2. Arizona doesn't always have nice weather. Haboobs, fog during the winter mornings/nights, monsoon season with heavy rain and hail. There are plenty of weather conditions in Arizona. And if building the visual recognition is the easy part, what is the hard part?
3. Fair, but that is already something their parent company does as another business; Maps/Earth. There is nothing to suggest they CAN'T do this on roads they haven't seen before, but it's obviously far safer to do this on roads they know.
4. They partnered with Daimler to make self driving trucks who couldn't use Uber to move shipments. They aren't competing with Uber necessarily.
As for Tesla, there is nothing to suggest they are doing it the right way. They have had numerous self driving failures.
They just raised $3bn (the vast majority of which is presumably not spent), and piecing together other sources indicates Google put in a couple billion of their own.
This article claims they've spent $3bn on R&D:
https://www.caranddriver.com/news/a30857661/autonomous-car-s...
Google's approach requires a lot of mapping, yes. But they mapped damn near every road in the important markets to pursue maps, an opportunity two orders of magnitude smaller than self driving.
If their cars and software are both ready, the mapping won't be more than a small roadbump.
3. Sure, they use maps.
But do we know that part of their algorithm there aren't map-free techniques ?
1. Prioritizing safety over everything has kept regulators far away and there haven't been any mishaps that I'm aware of.
2. Using multiple high resolution sensor modalities (LIDAR, camera) has led to very accurate localization and environment understanding, evident in on board videos [1]
3. LIDAR costs continue to drop precipitously, it now seems quite possible that next gen Teslas may sneak a unit on board to improve localization and depth perception.
[1] https://www.youtube.com/watch?v=ZsfUmU30kWY, https://www.youtube.com/watch?v=qAZ6tJSj9T4&t