I guess it depends on the data available, but I don't think the hyperparameters are what's needed. You just need to see what the car sensed and how it responded. Then the other car companies can try to replicate the similar circumstances and see what their models would do.

> You just need to see what the car sensed

I don't think all self-driving cars have the same sensors.

If you have LIDAR model X, and they were using LIDAR model Y, will your system "magically figure it out?

If your car has cameras at 5ft high, and the data is from a camera 6ft high?

Sure, someone could release the data, but will it screw up your models more than it fixes them?

(I totally agree the data should be released, I'm just not sure other self-driving cars will directly benefit. Certainly they can indirectly benefit from it.)

Is the design of self driving cars so limited that we will have to go through this every time carmakers must redesign hardware and/or vehicles themselves? Will the experiences of a sedan be impossible to transport to the experience of a semi truck? And vice versa?

The idea you present is possible, but I have to wonder how viable it makes the idea of self driving cars.

when self driving cars becOme ubiquitous it will be thru a renting model and therefore the fleet can be updated accordingly by companies owning them.

I understand, but the error may be in how it interpreted what it sensed. This is callous language, but if the models interpreted the pedestrian as trash on the street, then how it responded (driving over it) is not inappropriate.

> I understand, but the error may be in how it interpreted what it sensed.

It may, it also might not. If sharing data fails, further methods would be needed - but it's a good start for figuring out what data should be recorded for comparison. If the data is entirely incompatible, then we should have regulation to require companies to at minimum transcribe the data into a consumable format after an event such as this.

> ... if the models interpreted the pedestrian as trash on the street, then how it responded (driving over it) is not inappropriate.

If the models saw anything at all, they should not have driven over it. Even human drivers are taught that driving over road debris is dangerous. At minimum it puts the car/occupants at risk - in extreme cases, the driver may not recognize what it is they are driving over.

If this isn't a case where the car was physically unable to stop - it's more likely the telemetry didn't identify the person as an obstacle to avoid at all.

The idea is to come up with a certification test suite, and the cars have to pass it. Add/modify the tests as experience requires.

I would expect nowadays you could fairly trivially produce near life-like footage and allow self driving cars to 'play a video game' that tests tens of thousands of situations as a pre-requisite for certification.

The great thing about computer generated test cases is you don't need real footage of every hypothetical awful thing that could happen i.e. a truck losing control and rolling over sideways. These could be a stage 1 test -- a pre-requisite to real testing. Like the hazards test before you are allowed to sit the drive test with a real drive test employee [1] to make sure you're not a retard.

[1]: https://www.vicroads.vic.gov.au/~/media/images/licences/hpt_...

That may lead to algorithms being optimized for the “video game” instead of real life.

Next step is you put the system on a test track where you throw a wide variety of cases at it.

What happens if it uses lidar (or any other sensor)? I don’t think it’s easy.

Airplane certification uses a combination of simulator exercises and real world flying through prescribed maneuvers.

Mumble mumble Dieselgate.

I think there is likely a common need for a set of standard training models. This may hinder innovation for some time, but it's a cost we should accept when releasing a potentially dangerous technology to the public. It would have the added benefit of multiple companies contributing to a common self-driving standard which could accelerate its development.

That being said, when the first real cars were introduced to the world and later improved upon there were many more fatalities than we're likely to experience with self-driving technology.

As commented in another thread here [0], HAL at Duke University [1] is putting forward proposals for this.

[0]: https://news.ycombinator.com/item?id=16620439

[1]: https://hal.pratt.duke.edu/research

I'm sorry, but this is silly.

Every technology is dangerous. Every technology costs lives to some extent when spread across billions of people. I'm sure forks take more lives each year than self driving cars.

Weigh this against the potential lives saved. I posit you'd be killing more people with drunk drivers by slowing innovation than you'd be saving via luddism.

The first real cars, dangerous as they were, were far safer than horses.

Just sensor data to replay the scenario as fully as possible should be sufficient. Whatever it looks like to humans, there's clearly something in there that can be difficult for AI systems and so everyone should be using it as a regression test.

They need to share situations that made systems fail, and ensure that it doesn't happen with their specific system.