You have two choices:
o have a sensor that gives you depth directly (ie laser, radar, or lenticular array) o try and infer depth from stereo cameras
Lidars are expensive and power hungry, Radar is cheap and mature, but doesn't give you anywhere near as much resoultion
lenitucular only has a tiny range. There are other time of flight sensors, but they are either not production ready, expensive or both.
What Tesla have chosen to do is kinda try and merge radar and monocular object detection to give a higher frame rate depth estimation of objects. However its expensive to develop, unreliable, and terrible in corner cases (ie, if sees an unknown object, it can't place its depth.) Now humans can do this, because we've had years of training, Tesla can't.
Tesla's fancy cruise control is dangerous. Its auto drive stuff its trying to develop is even more dangerous. Instead of finding the sensors and processing spec needed to drive safely, they are trying to use the sensors and GPU they already have. Its not going to work and is unsafe.
There are, and I wouldn't trust them with life critical stuff. The more accurate ones are not realtime. The hardpart is that they are very noisy. unknown objects wobble about in depth considerably. You need to do lots of filtering to get useful results, which eats into time.
It is very much an unsolved problem. Its one that I'm partly working on now. However for the device that I"m partly working with, monocular estimation is far too power hungry, noisy and generally shite.
> Furthermore, most companies use various focal length lenses.
no, tesla _rely_ on having a wide, medium and zoom camera. However they blended into the same effective sensor to give a better chance at tracking objects. I bet you a $10 that if you block one, the whole system turns to shit.
> The real problem lies in sensor fusion.
Sensor fusion is trivial. Its understanding what they sensors are telling you, thats the hard part.
for example, when trying to turn across oncoming traffic, all the sensors will tell you that stuff is coming towards you, what they won't tell you is if its safe to cross. Thats the hard part. Given that tesla can't accurately place a car on the road yet, they can't safely cross traffic.
Sensor understanding is the problem that is really unsolved. Depth estimation with stereo cameras is 85% of the way there, monocular estimation is no where near that level, simply because robust object recognition isn't going to be a thing for at least another 6 years.
If computer vision fails at all to recognize depth (semi truck with clouds painted on it isn’t recognized) or misclassifies whether something is near or far that’s much more dangerous.
A human on the other hand can recognize another human with near perfect accuracy.
Humans can mostly do this. Like 90% of UFO sightings are people seeing the moon and getting confused and thinking it is a nearby object following them.
A camera feeding into phenomenally sophisticated AI trained on all sorts of weather conditions with windshield wiper smear and glare and everything else, which can turn those horrible images into a picture of what actually lies ahead, is an object sensor. Those aren't good enough yet.
IMHO AI to analyse the data is what’s missing - we have cameras that are just as good as human eyes, so we know for a fact that the current sensors are enough to drive in general weather conditions.