> They showed that vision could already outperform radar.

important to note that much like there is a wide range from bad cameras (“filmed with a potato”) to high resolution cameras, there is a wide variety of radars with different capabilities.

the radar unit in Teslas was pretty limited (basically designed for adaptive cruise control), and they showed that vision could outperform that radar (and have no interest in exploring non-vision approaches because “humans can drive with just eyes”)

Yeah, (un)fortunately compared with cameras, eyes have really high resolution and (most relevant) a truly excellent dynamic range.

In fact, there are many many many more signals that your eyes give your brain than just a 2d grid of pixels, which is why cheap off the shelf cameras slapped all over a car CANNOT do what two eyes can.

Your eyes also give coarse distance info by having to adjust focus, slight parallax (which depends on a HUGELY POWERFUL and pretty good edge detection and object classification system), an ability to perceive distance by the slight difference in orientation your eyes have to look at something in 3D space, some information about YOUR OWN head's orientation and movement, and a brain with a rough approximation of a simulation of the world around you to constantly check/verify any info against.

The brain seems to make most of the resolution up, it seems the eyes really can see only a small area. The rest is made up by object permanence. https://www.shadertoy.com/view/4dsXzM

Modern sensors combined with computational photography does seem to be changing this though.

For example, I now use my phone camera to see where I'm going at night if it's dark, and to read distant text. This is despite my (corrected) eyesight being better than 20/20 and I have better than usual night vision.

The eyes are fantastic organs, but the thing that makes them exceptional is the brain. Computers seem to be catching up with that.

Flashlight's have come a long way. I much prefer that to using my phone, especially on Android where opening the camera can be finicky and turning on the lamp requires a ridiculous amount of steps compared to the iPhones. Especially handy if out in the cold with gloves. Armytek prime c2 pro is pretty great.

I do love a good torch, and I keep one on me most of the time.

However, I meant non-illuminated phone camera. The new low light sensors combined with AI is incredible. They can see things when it looks pitch black to me.

Also not sure how opening a camera is finicky, just double tap the power button and it opens for me, and turning on the flash takes two taps - though I rarely use the flash.

Sounds like I need a new phone then. I'm still on the pixel 3.

Double tapping the power button on Android has been the shortcut to open the camera for quite a long time. It should work on the Pixel 3.

Why do you have to open the camera in order to use the flashlight on an android phone? The button to turn the flashlight on is in the quick settings panel which can also be opened from the lock screen.

On my older Motorola it was even a gesture of shaking it twice like a hatchet would turn the flashlight on and off.

How I miss the Moto X 2014, probably my favorite phone of all time.

> eyes have really high resolution

That's not actually the case, from what I heard. Most of our peripheral vision is smudged 720p at best, and we can only see a tiny focused area at slightly beyond 4K density. We also can't look at two things at once.

The dynamic range part is definitely true though.

Eyes have other special properties (e.g. saccade) which contribute to their unique capabilities in this space.

I'd love to see the convincing data RE: radar - they're using the same radars as every other car that has emergency braking from the car in front of the car that's in front of you. I've not heard of many "phantom braking" sessions from these other vehicles - I may have missed something.

Allow me to take you for a drive in my Volvo V90. I turned the auto-braking feature off entirely because it spits false positives like crazy. Parked cars, signs, birds, cars travelling through roundabouts across my nose, nothing at all - they all trigger it, and the car stomps the brakes extremely hard. The 2017 VW Golf it replaced was FAR better - maybe two false positives in 5 years. The Volvo? Daily.

Model 3 vision only does the same.

How often would you say you get false positives with your Model 3?

I have a model X raven (2019) and model 3 awd (late 2021), both have radar and hw3, and both have the update that has disabled the radar.

Pre-patch there was maybe a phantom brake (hard brake) every six months, but lots of small slowdowns where you could almost sense the car got conflicting input (once pr 100 km maybe).

The latter is gone and I have not had any phantom brakes yet, but I've had a slow down where a bus went into my lane. The cars behaviour in stop and go traffic is also much better now.

The only regression I've found is that it's a bit more confused in a local spot where the road splits in two in a turn to the right. In the past it went into the right lane without much fuzz, but now it is confused for a second or two before it decides on the right lane.

Tbh, as a daily user of these cars I think the no radar update was an improvement.

  > The only regression I've found is that it's a bit more confused in a local spot where the road splits in two in a turn to the right.

Since the issue is reproducible, you should find a bug on it. Tesla is responsive and this will help everybody.

Every single time I use autopilot. I stopped using it.

Also, my experience with the Volvo one (on XC90 tho) was that it would happily plough into cars standing at a red light in front of me if I would let it.

TM3 has never had this issue for me.

> birds

Birds you're about to hit with your car? That sounds like a great feature.

Birds flying across the nose of the car half a dozen or more metres away. Not a chance of hitting them.

But we already have a deal with the birds...

Where do you drive? I feel at least some of it depends of driving style and culture... Rental MB I had in Spain had phantom braked few times a day until I eased off. Other cars in LT and NZ - maybe once a month or less.

And by phantom braking I mean emergency braking a bit too early - i.e. I am perfectly aware I am too close to a car in front, I have foot on brakes and I see how traffic flows, yet the car brakes.

How much time would you have to brake, if the car ahead came to a sudden stop (for example, due to a crash)?

If less than one second, you are almost surely tailgating.

In Sydney, Australia. I’d say I’m quite a conservative, safe driver (I have kids). I don’t tailgate, and actually I’ve never had it false positive on a car driving in front of me. It’s typically on stationary objects, or small moving objects. Or on nothing at all shrug

Karpathy covered it during a conference keynote here: https://youtu.be/g6bOwQdCJrc?t=1368

It also is pretty common with other vehicles, they just don't get as much press as Tesla:

https://www.rivianownersforum.com/threads/phantom-braking-an...

https://www.fox43.com/article/news/feds-try-to-determine-why...

Hypothetically, Tesla would spin this story differently in no time if they are making great strides in a cheap solid-state radar.

For now, though, it looks like Tesla is building a case after they decided to do away with radar. I recall EM saying that it looks ugly and expensive too. I really got confused when Telsa came up with this patent though https://twitter.com/iamkellex/status/1534240730633236480?s=2...

No one sensor will work for all use cases; each has pros and cons. Radar really shines with depth sensing; it can cut through rain, fog, and snow like a hot knife through butter, much better than pure vision-based systems. At the same time, it seems to fail during harsh breaking and maybe a few other scenarios.

The ability to cut through rain, fog, and snow is precisely why so many companies use radar based systems - that's arguably when they are the most important and useful.

Leaning further into their vision systems when their vision systems cause their cars to slam into emergency vehicles that aren't even in travel lanes, and to rear-end tractor trailers, isn't encouraging. Musk is placing the rest of us at risk and I'm sick of it.

Have you seen a recent video of their most recent vision stack in use by a customer?

Nope! Why do you ask?

However phantom breaking still happens regularly (and perhaps more often) with their vision system. Shadows really mess with it.

So I’m not sure that’s very compelling that radar is somehow making things worse for them.

I get phantom forward collision warnings on my car quite often, a 2015 VW Jetta TDI SEL. There are a few places where it triggers ~1/3 of the time while driving on a clean road with no other cars around in either direction.

The system doesn't trigger breaking just a warning alarm beep and dash screen warning message, but all these systems are unreliable at this point, even the newer ones have these same false positives and false negatives.

It does make sense to err a bit on the side of caution for warnings. With warnings, a false positive is better than a false negative. It’s better to occasionally annoy the driver (within reason) than fail to warn for an accident.

This isn’t the case with braking. You want to have a high degree of confidence in that scenario because you don’t want to cause an accident. It’s better to react later to an impending collision than it is to cause an accident that would have never happened.

I got a ton of phantom braking just two weeks ago when I rented a 2020 Toyota Corolla with this feature. I've also gotten it in a 2019 Tesla Model 3, of course.

Anecdotally I've been driving a 2020 Toyota Corolla Hybrid for around 2 years now and haven't ever experienced phantom braking. I have driven it all around Australia (Melbourne, Sydney, Coffs, Brisbane) and the driver assist features have been nothing except rock solid reliable.

I have noticed though that it is very timid when people are leaving their lane, and it takes a few seconds to decide that a car has fully left its lane. It will brake as though the car is still in your lane well after it is gone. I'm not sure I'm going to hold it against Toyota, because I'd prefer it to be timid in case the other driver does something unpredictable like change their mind. If you press the accelerator pedal lightly when you notice someone leaving the road, the software will cancel out the braking.

I've only had phantom breaking with my 2019 Camry once or twice.

I drive a 2018 Hyundai IONIQ EV. The forward radar is hit-and-miss in heavy rain: the car will often barrel quite happily into stopped traffic when it’s raining hard, requiring a stomp on the brake pedal.

Just a couple weeks ago I was driving on the motorway around 4am, no other traffic for miles, and the forward collision alert sounded and the car slammed on the brakes for ~quarter of a second and then released as if nothing had happened and we merrily resumed our journey. Having the brakes slam on at a smidge over 100 km/h gave me a sore shoulder and quite the rush of adrenaline, quite the rush at 4am!

So yes. The radar does have a mind of its own.

Now imagine that in the snow, when going around a corner on a cliff.

Welcome to Canada 11 months of the year, with wolves waiting at the bottom of the ravine, as you crawl, injured, out of your car.

And no, wolves will not be appeased with either maple syrup or poutine, trust me I know, I've tried.

Not braking per se, but on all cars I've driven recently (ie some Seat Alteca, our BMW F10 5-series and maybe 2 more a bit longer ago) there is sometimes collision warning in situation where no collision is about to happen.

Seat was pretty bad in this, had it rented for 2 weeks in Sicily few weeks ago and first few times all screens start flashing like crazy in normal situations is very distracting (since I wasn't sure what the heck is happening, but it looked sinister). BMW is pretty subtle with this, and only happened few times in past year.

I imagine if this warning system would be also connected to breaks, bad things would be happening.

They were using the radar for lane guidance. That's why they had to have a whitelist of locations where the radar was false triggered by the roadside environment. It never actually worked properly and they just papered it over with a hack to temporarily blind their input.

How does radar for lane guidance work exactly?

I'm pretty sure all of the phantom braking events due to underpasses were vision related, given that radar is essentially blind to the road.

Radar for lane guidance? Uh, are you sure?

None of what you have said is true.

> They showed that vision could already outperform radar.

Except on emergency vehicles parked in a lane at an oblique angle, which Teslas did not recognize, and plowed into at speed. I wonder what unknown secondary effects this change will bring.

> With these parking sensors it's different. They have no current alternative and it will cause a pretty significant loss of functionality. Disappointing move.

I'm especially worried by dark garages / parkings. Tesla cams have already enough problems when outside is too dark.

Some pretty convincing data that they handpicked to make themselves look good and pretend they don't need a radar, a lidar and that vision only is good enough.

Perpetuating a pattern of lies from Tesla, ever since they started on self driving.

You don’t need to take Tesla’s word for anything. Their latest vision stack is in the hands of ten thousand customers, some of whom regularly post their drives on YouTube.

And that regularly demonstrate catastrophic failures, yes, I know.

What's your definition of catastrophic?

"Would cause an accident without intervention".

More data shouldn't actually hurt, it can be use as bayesian prior on the camera data when the vision stuff is uncertain if nothing else.

But that’s the thing. Tesla wasn’t able to tell when the radar was “uncertain”. So when do you trust one data source more than the other? If it was so easy to label misinformation as such, the misinformation would probably not have been communicated in the first place. Tesla explained this at AI day 1.

If the vision stack is just below its certainty threshold that a car is coming across at 63mph at some angle and can't quite decide to take action on its own, and at the same time radar indicates a car is coming across at the predicted angle and speed, it should push it over the threshold, even if it might get false positives from overhead bridge traffic Lining up with the vision estimate makes that less likely, and the vision stack itself can also be used to exclude data that might be from an overhead bridge by detecting that there is no bridge nearby.

No sensor is certain and can be blindly trusted - the data is going into a neural net anyway so the computer can be trained to understand what the most likely ground truth is given all sensor data.

Radar has a longer wavelength than visible light. Thus making it better at some stuff and worse at others. It's a crime to not be using as much of the EM spectrum as possible for an application like this.