I get annoyed with statements like this because technology changes and advances so quickly, and Tesla has made substantial technical leaps in this field of machine learning. They have the state-of-the-art vision -> voxels/depth models and are only improving.
Waymo, who use additional sensors like lidar, have a driverless taxi service which needs no safety drivers.
The evidence kinda speaks for itself, IMHO.
[1] https://twitter.com/elonmusk/status/1740641473849352450?s=20
> Tesla, who use cameras only, have not demonstrated full self driving
There are entire youtube channels with hours of continuous video showing Teslas driving around SF, but also other parts of California, with no human intervention.
Waymo is geofenced, but within its geofence it requires zero human intervention. Tesla on the other hand is famous for mistaking the moon for a traffic light. Saying "Tesla has so many miles on YouTube" is hilarious because first of all there are channels with lots of Cruise & Waymo footage too, and more importantly it's not the # of miles that matters, but the # of non-trivial scenarios you can handle.
Btw Waymo is nowhere near achieving unit economics. Their cars cost like 5 times what Teslas cost, and the sensors require a lot of upkeep and maintenance.
It's probably not accurate to say that the remote operator is "steering" though. It's more like pathfinding.
<photo of construction zone> Can I drive through here? [Yes] [No]
When this is happening, the car is stopped and lets the passenger know that it’s reaching out for remote help to figure out what to do. For me this has happened two times across my 125 Waymo rides (571 miles) so far, and was resolved in under 20 seconds. Though I must say, 20 seconds feels like ages when you’re in the car and blocking traffic!
Eyes which are orders of magnitude capable than the best cameras, and Teslas come with mediocre cameras, not the best. Eyes which are connected to a brain, and ML is a looooong ways from rivaling that.
> That's a reality of life that robotaxis do not need to solve.
Robotaxis do not need to account for things jumping out unexpectedly in front of them?
My brain tries to hides all this from me, and makes me think that I see the world in glorious 3D technicolor all the time, but that's a lie as revealed by the many amusing optical illusions that have been discovered over the years.
Meanwhile, today I used ML that knows more than me, can think and type faster than me, which is a much better artist than me and which can read and react far faster than me to visual stimuli. Oh, it can also easily look in every direction simultaneously without pausing or ever getting distracted or bored.
Somehow it doesn't feel like I have a big advantage over computers when it comes to driving.
The maximum dynamic range of the eye is ~130dB. It's very difficult to push an imaging system to work well at the dark end of what the eye will do with any decent frame rate.
It's not as different as it used to be, but even so: the Mk. I eyeball does pretty damn well compared to quite fancy cameras.
Depth sensing is again, estimated or using time of flight sensors which is pretty much short-range lidar. HDR is used already in AV perception, but still loses to your eyeballs in dynamic range and processing time.
Eyeballs have high dynamic range but with high mode switching times. Walk from a bright area to a dark area and it'll take seconds for your eyes to adjust. Cameras are so cheap you can just have a regular day camera and a dedicated night vision camera together, switching between feeds can be done in milliseconds.
Robots aren't humans. You need accurate depth perception to maneuver a robot precisely, and you need ground truth depth measurements to train learned depth perceivers as well as to understand their overall performance. Humans learn it by combining their other senses and integrating over very long time using very powerful compute hardware (brain). To date, robots learn it best when you just get the raw supervision signal directly using LiDAR.
> Walk from a bright area to a dark area and it'll take seconds for your eyes to adjust
You do realize cameras have the same issue, and that HDR isn't free / is very computationally intensive?
Your brain is _really really_ good at surmounting challenges including many that you did not mention. We don't know how to get close to this in terms of reliability when using cameras and ML alone. Cameras and ML alone can go very far, but every roboticist understands the problem of compounding errors and catastrophic failure. Every ML person understands how slow our learning loops are.
Consider that ML models used in the field have to get by with a fixed amount of power and ram. If you want to process time context of say 5 seconds, and with temporal context 10Hz and with resolution 1080p, how much data bandwidth are you looking at? Comparing what you see with your eyes with a series of 1080p photos, which is better? Up it to 4k: how long does it take to even run object detection and tracking with a limited temporal context?
Your brain is working with more temporal context, more world context, and has a much more robust active learning loop than the artificial systems we're composing today. It's really impressive what we can achieve, but to those who've worked on the problem it feels laughable to say you can solve it with just cameras and compute.
There are plenty of well respected researchers who think only data and active learning loops are the bottlenecks. In my experience they're focused on treating the self driving task as a canned research problem and not a robotics problem. There are as many if not more respected researchers who've worked on the self-driving problem and see deeper seated issues -- ones that cannot be surmounted without technologies like high fidelity sensors grounded in physics and HD maps.
Even if breadth of data is the problem and Tesla's approach is supposedly yielding more data -- there is also the question of the fidelity of said data (e.g. the distances and velocities from camera-only systems are estimated and have noiser gaussians than ones generated with LiDAR). If you make what you measure, and your measurements are noisy, how can you convince yourself or your loss function for that matter that it's doing a good job of learning?
It's relatively straightforward to build toy systems where subsystems have something on the order of 95% reliability. But robotics requires you to cut the tail much further. https://wheretheroadmapends.com/game-of-9s.html
Secondarily, there is no guarantee that the amount of processing is enough, because the extant human systems use much more.
“Cheating” by using more sensors to simplify out complexities and to cover for the shortcomings of other sensors in the suite seems wise.
Also, "cheating" is just a necessary step to build baseline metrics. You need ground truth.
It may very well be the case that cheating is needed to generate the training data necessary to stop cheating...someday.
IMO the bar is well above human-level if you actually want to attain the level of trust necessary to remove the steering wheel from a car.
For one, frame rate and processing rate on human eyes is way higher than cameras. Dynamic range is another. Also, Cruise and Waymo are some of the only companies that have hard internal data / ability to simulate how well their safety drivers do, and in the very same scenario what their software driver will do. Without LiDAR you can't build that simulation, and once you have that data if you continue to use HD Maps and LiDAR there's probably a good reason.
> Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.
Robotaxis need to avoid any accident that a human would be able to avoid.
> IR and thermal sensing are unnecessary if the bar is human level
See, you could say this if you had some data that showed that incidents per X miles (when the vehicle is driving at night) is sufficiently low, + if the software passes some contrived scenarios to gut-check its ability to see in the dark with the necessary reliability. But you don't have that data, do you? Someone has it though :) and I'd argue regulators should have it too.
I don't think it's exciting to say that you must have theoretical parity with something to use it for this use case. Tesla's solution monitors ~6? cameras at once with accurate depth in each. That's 6x more views than a human can see. I wish people would stop comparing apples to oranges.
> Robotaxis need to avoid any accident that a human would be able to avoid.
I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.
No, the depth is estimated. It's not accurate, at least not in the way you need for L4.
> I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.
I was just clarifying what the bar is. The bar is that avoidable accidents need to be avoided. Nobody will get mad if a plane crashes due to unavoidable circumstances (freak accident where two engines go out due to bird strikes or something). People will stop flying in the plane when it becomes clear that the airline is not doing everything it can to avoid fatalities.
IR and thermal sensing are unnecessary if the bar is human level and neither is the lidar. The point is overused, but humans rely on two eyes in the driver seat. I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5.
Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.