Nobody (well, theoretically someone, I'd like to be proven wrong) has a self-driving algorithm which is formally proven. If we can't get, to use your example, web engineers to formally prove their web applications, when they have all these structural advantages, then how can we expect something unproven (har, har) to be formally proven? So there needs to be an extensive QA test suite.
And for all of these other relatively-mature engineering sectors like web engineering, how many of these companies are really investing in QA? In strong and comprehensive test suites, in security reviews, in performance testing and UX scoring before release? Almost nobody, right? I mean, the only reason why society can get civil and biomedical engineers to submit to strict quality control is through regulation, but no regulatory body has any real idea of how to apply that to software. And autonomous driving systems are supposed to be different?
Mythical Man-Month came out 38 years ago. The Pareto Principle, not so much as being explicitly a 80/20 split but about how the vast majority of the effort is needed to achieve actual completion rather than just a proof of concept, was originally coined in 1896. Suddenly it doesn't apply to Uber?
Is it really so surprising that it's so difficult to find senior practioners in these areas when few companies are committed to that level of quality, and the companies which are tend to be so successful that they can afford to pay compensation at rates so far above the rest of the market so as to ensure that few of their senior engineers ever leave?
Society usually doesn't need a formally proven cat pics sharing app, and most web apps, including facebook, are that level of application. They would rather pay a cheaper price than to have a perfect expensive thing.
You'll also notice that as organizations get bigger, they do invest more in testing, QA and regression detection.
Needless to say, this is expensive. I’ve heard estimates that it costs roughly $1000/line of code.
The point is that there are methods to increase reliability, but there isn’t the financial incentive to do so.
If there is only a single developer working on a project, that developer can manage much much more code if there are solid tests behind it than if she was just winging it with every commit.
At some point the developer starts to forget the code they write and starts relying on the tests they built long ago.
I always assumed people would integrate an external simulator like CarMaker rather than build their own.
I'd be exceedingly interested in hearing about your product when it comes available (although my interest is in simulation for maritime autonomy rather than automotive).
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.