The thing is that it doesn't really matter. There are only 2 options here. Either, the car should have seen the woman and taken corrective action or it was driving faster than it can see. Just because the speed limit says X doesn't mean that's the speed you should be going. Often, actual conditions require driving slower.

Well, I do feel like I've been had after that and a few other videos, so thank you for that.

My point though, is that I don't want self driving cars to defeat simulations or tests, I want self driving cars that can react to real driving conditions.

But how else do you imagine scenarios like these are tested? Normally there aren't a lot of people willing to jump randomly out in front of cars to test their reaction.

I really don't know. Even with all the hours of recorded road time by Google/Uber used as a testbed for proper driving conditions, most machine learning systems still suffer from figuring out ways of cheating[1], and in those cases it could be fatal.

[1] "The program even learns how to take advantage of bugs and glitches, like timing jumps so that Mario begins falling again at the exact time that he makes contact with a Goomba. Mario's invincible when he's falling, so the touch kills the Goomba, not Mario, and it gives him a further jump boost." http://www.wired.co.uk/article/super-mario-solved

That's a problem for the test design. Maybe we can develop an approach that uses more randomness to test these systems.

But I dont see how you can avoid having a simulation. These events are just too rare to wait until they occur in real life.

It is not ML deciding where to turn, when to slow down.

I think comparing this to human drug trials is an appeal to emotion.

I also think you're saying that bad tests are better than no tests, which I don't agree with. These would be bad tests because they wouldn't constitute the complicated reality of driving, they inherently only summarize the output of such a complex system.

You seem to have confused the concept of bad tests with incomplete tests. Asking to stop the car when pedestrian shaped obstacles appear seems like a bare minimum that we should expect from an AI before letting it drive on real roads. We can't test every driving situation in a lab, but that doesn't mean we can't test some of them.

If you think it’s an appeal to emotion, make a real argument in that direction, don’t bluntly state the contents of your head and move on. As it stands, you just appear morally bankrupt. I’m strongly inclined to believe that’s the case, and your inability to make a cogent argument makes me wonder about what else you’re lacking.

By the way, you may be misunderstanding “appeal to emotion” as “having emotions” and that’s worrying. I can only hope your attitude is literally pathological, and not just your personality. Either way, please never work near people, or I’m a critical system effecting people.

It would be morally bankrupt to rely entirely on a quantitative test for something that is qualitative, I don't know how much clearer with the previous comment I can get?

We've banned you for repeatedly violating the site guidelines.

https://news.ycombinator.com/newsguidelines.html

You're missing the fact that any time someone takes a drug, they're essentially testing it against their unique body chemistry. The same as self-driving cars in unique situations.

The only difference is the car feedback loop is way better instrumented, and doesn't have to rely on mass statistics to inform recalls.

> Half of all humans who have ever lived, died from mosquito-borne illnes[s], about 5,000,000,000 human lives.

Your math doesn't work.

The current population of the world is between 7 and 8 billion.

If half of all humans who have ever lived were 5 billion, the total number of people who have ever lived would be 10 billion, of which at least 7 billion are still alive.

That leaves less than 5 billion who could possibly have ever died for any reason, much less mosquito-borne illnesses.

“Modern” Homo sapiens (that is, people who were roughly like we are now) first walked the Earth about 50,000 years ago. Since then, more than 108 billion members of our species have ever been born, according to estimates by Population Reference Bureau (PRB).

Source: https://www.prb.org/howmanypeoplehaveeverlivedonearth/

I imagine just rolling some objects across the road or some self-riding bicycles should be enough.

But what if the self driving cars fail the simulation? The simulation would be the first thing the car needs to pass, if car passes the simulated, then special tests, then you get approved to test on public roads with a security driver, I have no idea when is the time to test without a driver.

I know this is going to sound uncharitable and I acknowledge that any death is tragic but...

This person carelessly crossed a busy road at night! The victim practically did commit suicide. She could have just as easily been killed by a human-driving car, just like thousands of pedestrians do every year. Why are we so quick to try to blame the car? At least the car was in the right place.

The car was not in the right place, or it would not have hit her. In the state of Arizona a pedestrian has right of way in front of any vehicle, and the driver must yield to any pedestrian on the same side of the road.

Edited to add: We don't know anything about her physical or mental condition that might have made it difficult for her to perceive the car or the distance when she was crossing, so to assume that she had perfect perception and chose to cross in front anyway assumes facts not in evidence.

Coming back to this, I was wrong about the details of Arizona law. They require pedestrians to yield to drivers outside of crosswalks, marked or unmarked.

https://www.azleg.gov/ars/28/00793.htm

However, I still think Uber violated 28-794.1 and .3.

  Notwithstanding the provisions of this chapter every driver of a vehicle shall:
 
  1. Exercise due care to avoid colliding with any pedestrian on any roadway.
  2. Give warning by sounding the horn when necessary.
  3. Exercise proper precaution on observing a child or a confused or incapacitated person on a roadway.

https://www.azleg.gov/ars/28/00794.htm