More importantly, they're the only ones accessible to the average consumer. Waymo/Zoox/Daimler all have equally if not more impressive systems.
A real issue with tesla is that they want to be vision-only, which is going to make getting to level 5 first almost impossible.
BTW - knowing where the moon is happens to be an extremely solved problem.
It's true... but I don't think that's how Tesla would solve it. Their goal is to create a neural network "driver" which can drive in any place even if it has never seen it before. They'd rather teach their neural network that the moon and stoplights are not to be confused visually. Thought I suppose in searching for training examples they could use the known position of the moon for approximate labeling.
Isn't that impossible considering these networks need training and therefore have seen everything before?
But with the Tesla, it is learning to drive in general. It does not need an HD map of a fork in the road to understand how to navigate it. Just as a person who learned to drive in California will have little trouble driving in Florida, a neural network that has learned to drive on a million intersections will be pretty good at navigating most intersections. Especially because the corner cases will stand out and become integrated in to training. So it may see many intersections, but it will generally know what to do with one even if it has never seen it before.
Though I would suspect that the competitors are perhaps using HD maps to jumpstart a system that long term would behave more like the Tesla one. Mapping every road is a lot to ask.
One actually really important feature of these systems is how they handle failure. If the car gets confused, how does it handle it?
Permanent yield signals are often only one flashing yellow light. Crosswalk signals are often only flashing yellow lights when active. Temporary construction barrier signals are often only yellow flashing lights. Fire station signals often have only red and yellow lights, no green. Even when all three lights are present, they may also be oriented horizontally, or in triangular shapes.
Which isn't to say there isn't more context to learn from, but just about the only true unifying trait among all these indicators that you should perhaps pay attention and slow down is a bit of yellow light. It need not even be circular: yellow arrows are far from uncommon, including "straight ahead" yellow arrows for intersections where turns are forbidden.
But the big thing is that their autonomy computer can be programmed to look for odd scenarios and send them back home. Tesla uses their fleet of hundreds of thousands of cars to collect edge cases like this, and then they have a kind of compartmentalized neural network system that breaks apart disparate tasks. With their collected examples they can create unit tests to ensure that the moon stops activating the stoplight detector. Once trained, the unit tests presumably help ensure they don't end up with future regressions.
So basically every time you see a Tesla do a weird thing, there is a good chance it will stop doing it soon enough. At least if it's hitting hacker news.
Tesla’s vision/ml systems are amazing. I would love to learn more about how unit testing for this type of error is done. Without some intermediate semantic representation, I don't see how these large, multi-head, end-to-end systems can isolate and regression? System tests are maybe possible, but it's unclear how well a system test would generalize to related, but unseen cases.