I suspect that we'll see a progression like:
1) Well-regulated urban and highway environments
2) Well-regulated rural environments
3) Moderately regulated urban and rural environments
4) Poorly regulated rural environments
5) Poorly regulated urban environments
And I think we'll probably be stuck on #1 for a while. I would also venture that most companies claiming to "be there" or "almost there" are referring to #1 as well.
I also think that it's possible that politicians may try to legislate a transition up that list as that first case starts taking off (greater traffic enforcement, better signage/infrastructure, more rigorous driver training, etc).
Pretty sure you can't legislate those out of existence. And sadly I think the fact that these vehicles are being developed and tested in Silicon Valley is either a cause or an symptom of their probably inability to deal with those kinds of conditions.
Company that has all the roads and surroundings scanned and knows exactly where the car is at all times is at a huge advantage.
Partially observed world gets your head hurting. Not modelling other cars practically makes everything useless.
If you can't hallucinate everything around you as close as possible, the problem gets very very difficult.
I'm pretty sure there are companies close to solving the problem, as they have more data and some also know practically every road in the modern world.
Modelling and tracking other cars is a bigger issue, including pedestrians.
Aren't there very reliable ways to model other cars? Something as big as a car can't move very unpredictably in a millisecond-to-millisecond basis. And at the end of the day, all you need to do is be able to model the behavior of your car.
If car in front of me starts to swerve over on a collision course with my car, my car can note the probability of a collision a long (computer-long) time before it really needs to do anything about it. Knowing full-well its own braking, swerving and acceleration capabilities at any given speed, it should be able to know when it _must_ take action to avoid the actions of another, and it doesn't even have to rely on reaction time.
No, I think the real problem here is for cars to do things like understand _context_ of a situation, like, for instance, how can it tell that there's a bus in front of me turning around that's going to be clear of my way in a moment vs. a tree that's been felled? That's something that computers are and will probably continue to be terrible at until they can closely mimic the capabilities of the human brain.
Unscented Kalman filters and particle filters are used for that - they are pretty good if you sample environment with >10Hz rate. Most of what you said has been taken car of pretty well, those aren't really hot issues anymore. Also, the thinking these days is that humans (even mediocre) can do a good job driving using eyes only and more robust camera solutions based on deep learning are here. Their problem is inference speed as HW is not fast enough to allow FHD or 4K input feed to be processed realtime, so imagery has to be downscaled significantly, leading to errors. Similarly, tracking other cars only depends on how big supercomputer can you fit into your trunk.
The whole conversation is unfortunately poisoned by endless wishful thinking instead of discussing hard evidence. I mean there is still a large proportion of people on here who think Tesla will somehow leapfrog Google because they have all those cars out there streaming the raw video data from their umpteen cameras back to Tesla HQ, utilizing an imaginary mobile communications standard that leapfrogs LTE by two orders of magnitude that they invented alongside their LIDAR-less L5 autonomous car software in record time.
My personal skepticism on this subject has made me very unpopular at parties (well I don't actually go to parties, but you know) where lots of techy types go. Also earned me lots of downvotes here.