There are over 200,000 cars a day driving at that stretch of highway, if a car crashes there once a week, then the Tesla Autopilot is an order of magnitude more dangerous than a human driver.

Edit: in a posting today, Tesla has said more: https://www.tesla.com/blog/update-last-week’s-accident but at the time of the earlier blog posting, Tesla says they did not know yet.

That's very curious wording. The important number would seem to be "millions of miles per fatality while Autopilot is enabled", not "millions of miles per fatality in Autopilot-equipped vehicles". The way it's phrased, I wonder what part of that 3.7x multiplier is unrelated to Autopilot and is just due to the car's high safety rating in a collision, recent release (no long tail of older, less safe models contributing to the statistics), and luxury status (more likely to be driven conservatively by old rich people.)

Notice it says one fatality per "320 million miles in vehicles equipped with Autopilot hardware." But, how many of those miles (or what percent of the time) does a Tesla driver use Autopilot? Also, maybe those good stats are due to the structural safety features and not the autopilot. It may be more fair to compare to other cars with excellent crash protection but no autopilot.

I think it would be a great service to the world to improve driving safety, but maybe we need to really start looking at the stats and get some more hubris as we transition to full autopilot. For example, require that drivers keep their hands on the wheel and eyes on the road. When more cars have autopilot, then mesh behavior between vehicles, and e-road features, then maybe we'll be more ready for driver-less cars.

Also note that this includes trucks and motorcycles, which have much, much higher fatality rates than passenger cars. Motorcycles are around 10x-50x higher fatality rate than cars! So already Tesla's blog is doing a misleading comparison to more deadly vehicle classes.

Additionally, there could be all sorts of other variables that make it hide even comparison. Are Tesla drivers physically comparable to regular drivers? Are they older? Younger? Both elderly and young kids have a higher accident rates. Are Teslas driven in rural areas and urban areas the same as regular cars? Because rural areas have a higher fatality rate.

So we need to make sure that the Tesla driver matches driving conditions of typical cars to make autopilot comparisons valid.

Right now, it appears Tesla's autopilot is a death-trap.

I agree with all the holes you poked in their stats, but then with the last sentence you just went way off the deep end. What does "death trap" mean to you? To me, it seems likely the Autopilot engaged is about as dangerous as disengaged.

Let me share an example. I had an accident in a Model S, front-end, which didn't cause a loss of power. Tesla called me a couple of minutes later, saying that they knew I'd had an airbag-deploying accident, and asking if I needed an ambulance. From the description of this accident, power to the Tesla was lost before there was time to phone home.

The obvious failure was the highway safety mechanism not immediately being replaced after the previous crash - that would have made the Tesla crash have gone from death to perhaps only minor injuries.

I would argue too there is a need for AI image analysis - could perhaps be crowdsourced - to analyze the structure/state of a highway or any road. It would of course need to be trained, however it could also likely serve as tool to improve road safety worldwide - bringing everyone in line with known best practices. In this case, it would have caught the improper safety mechanism not being replaced, and then that death would have been avoided; cost-benefit analysis says this system would be worth it as life is invaluable.

If Tesla/Elon leads this effort it would show them being proactive in improving future safety, accountable, and taking ownership. Why has no other auto manufacturer done so, why has no government implemented such a system? Well, because honestly we are stuck in scarcity only caring about our own costs, and so if the random person dies here and there, there's not a big enough ripple effect (unfortunately) to cause enough personal worry to cause change. Tesla, if wanting to posture as being the steward I think many of us hope they are, does however absorb these ripples that get associated with their brand - and therefore the responsibility does pass onto them, whether they agree to accept it or not.

All auto manufacturers would likely benefit from this, allowing their software to work better within certain expected constraints. Many more possibilities as I think of it, newly spotted damage or cracks in bridges, detection of debris on highway or excessive dirt on the edge of the road making the conditions more dangerous for emergency stopping/maneuvering, etc.

Edit: I added a few sentences, so the one upvote prior to editing, they didn't upvote everything I said though it's all in line.

I fully agree with you here. Urban areas are too dynamic to rely on static maps and local LOS only. This sort of data sharing is going to be a necessity, IMO, to achieve level 4 or level 5 autonomy.

Advantage being that if you can provide representative, simulated input then you can increase the training miles by orders of magnitude in the same amount of time, limited by computation rather than physical mileage.

Trying to use them to model real world scenarios would be useless in practice, due to the Ludic fallacy [0]. Real life is too complex to be modeled in any simulation.

[0] https://en.wikipedia.org/wiki/Ludic_fallacy

As for modeling real life with simulations, the data exists for every type of accident a human has encountered. If you incorporate such into a simulation, plus randomly vary every free parameter, then your simulation will cover more scenarios than any human driver can possibly experience.

Thus, simulations should be able to help an autonomous vehicle outperform humans by a large margin, which is the only goal that matters.

(Keep in mind that until they recover the logs, we don't even know if the autopilot was on.)

https://www.tesla.com/blog/update-last-week’s-accident

>In the moments before the collision, which occurred at 9:27 a.m. on Friday, March 23rd, Autopilot was engaged with the adaptive cruise control follow-distance set to minimum. The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision. The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken.

So, clearly, the AP is not capable of avoiding such obstacles yet, even when unobstructed. It also seems like a mistake to not slow down to a survivable speed when hands are off...

https://imgur.com/a/iMY1x

Why did you eat a bagel instead of eggs this morning? Usually there isn't a scientific answer.

In the present, AI models are already so complicated that it seems hard to get reproducible diagnostic results, short of just saving every frame of data that the car's sensors pick up. And for video that seems rather prohibitive. Imagine collecting all sensor data for each of those 200 trips per day along the entire stretch of road.

The original guys (McCarthy, minskey, etc) at least understand the gigantic complexity of the undertaking in so far as we understand how the mind works at all, and they never reduced things purely to classifiers. That’s like 0.1% of the job. The real researchers take philosophy of mind seriously — technologists want a quick and cheap solution that cannot exist.

Maybe you meant sapience or intelligence, but I wonder if people think computers won't be sentient (as in self aware) in our lifetime, even if they are still not able to match human cognition.

And yet all this complexity and purpose may potentially be simply the product of putting a large collection of matter in a heat bath for a few billion years.

Who says that this process can't be repeated via brute force simulation? That the emergence of complexity and life under thermal disequilibrium + sufficient number of degrees of freedom is actually a basic fact of the universe?

I don't see any rule saying why, with sufficient brute force computational power, we won't be able to set up better and better simulations that will allow for the arise of true artificial intelligences.

We think the brain has 100 billion neurons. Making those work is a very complex network, lots of chemical gradients, bio-electrical systems, cellular-level systems, and probably a host of other things we don’t know about. So it wouldn’t surprise me if for single neuron you’d need to model another billion things at minimum.

Now factor in all of the other input senses, and those 100B neurons have increased dramatically for I/O. Plus a multiplier to model everything going on inside a neuron. Then there’s also modeling the communication layers between, effects of myelin sheaths and all that.

So we’ve established that just to run a single “step in time” (whatever that means, as that’s a computing concept), we’re into probably billions of trillions of calculations.

Now what those model have to make sense and do something coherent. Every future step. So now you need some kind of meta model that is able to push this forward and continue in the right trajectory. Now complexity is dramatically even higher.

Now that we’ve somehow figured out how to model everything required for intelligence, we need to search that space through brute force, as you put it.

To me it seems like the time required will be astronomical — Sun might have blown up by then.

So why go down this route? Nature solved this in parallel relatively cheaply through billions of iterations and billions of permutations over tens of thousands of years.

Seems easier to simplify the problem down to more basic stuff that works well enough and iterate / combine systems. Either that or find a way to grow a “brain in a test tube” that you can hookup to a computer. There’s also always humans available at mass scale for pretty cheap across the globe — sophisticated intelligence built in.

It doesn't have to be. There are alternatives to that.

Methinks there is far further to go than we realise - especially with the current air of hubris around the entire endeavour.

>Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.

e.g. If an image (from the forward camera) returns the probability vector: {"stop_sign": .9, "green_light": .1} You stop the car 10/10 times.

Obviously you can set the RNG seed to be the same every time too, but even that only works if your system is wholly synchronous, which a car probably isn't.

Note that I doubt Monte Carlo methods are common in the autonomous vehicle space.

I wouldn’t be surprised if Monte Carlo techniques are useful for all manner of things related to ingesting sensor data on autonomous vehicles.

I'm not in the automotive space, but I'd be surprised if there were a viable self-driving car team not using Monte Carlo methods somewhere in the vehicle stack.

EDIT:

Yes he did and lost[0].

Jeremy Clarkson did another bit in The Grand Tour about the new Tesla and it was quite British[1].

[0]http://www.thedrive.com/sheetmetal/12536/remember-when-top-g...

[1]https://www.youtube.com/watch?v=_DiGQRaaHvg

It's a strange accident, especially because of the strangeness of tesla and the whole background story (broken safety barrier). I'm looking forward to the outcome of the whole story.

At the end I guess more than one party could've been a bad actor. Tesla, The guy itself and even that the road had no kind of warning (safety barrier not replaced or some kind of other warning sinces that it will be replaced soonish)

Ofcourse, this statement by itself doesn't absolve Tesla's software of any error. It's important to know the actual root cause before saying who/what is wrong or not, something which the HN community (and in general Internet/people) isn't good at.

As someone who works on software for one of the automakers, I also want to note that there seems to be a lot of focus on the software... but it's also very possible the problem was with the hardware: something in the vision or sensor system was incorrectly installed, defective, was calibrated incorrectly, lost calibration during use, etc.

I'm sure the software can always be improved, but if most cars are driving through this stretch without a problem and his always veers, that sounds like something unique to his vehicle which points to hardware in my opinion. In cars, the software can only ever be as good as the data it receives.

He owns the car/work in apple/drive the same road to work - for about half year (i.e. >100 times) and according to the article he has just about 7-10 veering incidents here. Like one or two per month.

That must be like half the bay area. I get that it's written as a token of pride for you, but maybe worth to take a step back?

> It probably means that the visualization system does understand such scenes and is able to navigate it.

I think there is a pretty good counter example proving that it doesn't.

Erhem: https://i.imgur.com/sY3Eeln.png

Scenario 1: 100 human drivers drive some distance. 2 bad drivers caused 150 accidents and both died in their last fatal accidents.

Scenario 2: 100 self-driving/autopilot-enabled cars drive the same road for the same distance. Everyone has one accident and one of them is fatal.

Statistical numbers: Self-driving/autopilot-enabled cars cut both the accident rate and death rate by half. Death rate per accident is reduced from 2/150 to 1/100.

Conclusion: self-driving/autopilot is better.

"Tesla also posted these photos that raise another important question: they show what's called a "crash attenuator" or safety barrier in the proper condition ... and the way it was the day before Walter Huang's crash ... collapsed after a different accident."

Did you see any news coverage of the previous accident? I know I didn't.