News for hackers. Stuff that matters.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ajross on Sunday November 10, 2019 @01:41AM (#21496759)
No, the appropriate standard for a self-driving car is the driver they replace. If they're safer than whoever would have been driving (be it on a saturday night or not) then they're a net win. Right now the average idiots are buying more Teslas than mail carriers and cab drivers, though that might change I guess.

Numbers are numbers.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ars on Sunday November 10, 2019 @02:00AM (#21496820)
If you like numbers, then you'll like this one: Humans achieve 7 9's of reliability when driving a car, as measured for fatalities by time, or 8 9's if you measure by miles.

I've yet to see a computer just stay up that long, never mind actually doing anything the whole time.

(If you use the collision rate instead, the numbers are about 500 times worse - but that's still quite a lot of 9's.)

People have this fixed thought that humans are terrible drives. They are not! Computers have their work cut out for them to exceed the reliability record humans have set.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by aidenn0 on Sunday November 10, 2019 @02:09AM (#21496847)
You are using the 9s term wrong. It is supposed to be used with unitless numbers. Not crashes per hour or per mile.

Silly example of why: humans are zero nines of reliability if you talk about crashes per parsec.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ars on Sunday November 10, 2019 @02:19AM (#21496879)
> Not crashes per hour or per mile.

But that's not what I measured. I measured safe minutes of driving as a ratio to unsafe minutes. Which is a unitless number.

For 7 9's I assumed a crash had a 5 minute lead-in of unsafe driving before the actual crash, and that average driving speed was 30 mph.

If you assume the bad driving is 30 seconds (for example the accident in Tuscon the Uber car saw the pedestrian around 30 seconds before crashing), then you can add another 9, making both figures 8 9's of reliability.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by aidenn0 on Monday November 11, 2019 @01:53AM (#21502757)
Thanks for the clarification. I haven't checked your numbers but they seem reasonable.

FWIW, I think deaths, regardless of fault, is probably the best number for an apples-to-apples comparison. If the Uber car were driven by a person without a dash-cam, then there is no way they would have been ruled at fault.

Similarly, I have been involved in 5 collisions (most were not my fault). Of the 5, 3 were reported to insurance and only 1 was recorded by the police (For 1 other the police were called, but the dispatcher (on the non-emergency police line) said "Are both cars driveable? (yes) Is anyone injured? (No), then don't bother us!" (By comparison, in California even a single car collision on private property must, in theory, be reported to the police).

Deaths, at least, will get recorded by the CDC, if nobody else.

[edit]

However, I don't think that deaths are necessarily the best number for considering "failures" all deaths are in some way a failure, but most failures do not result in deaths. Nobody even had to see a doctor for the 5 collisions I mention. I've fallen asleep at the wheel and crossed the center-line without getting in an accident, &c. humans make a lot of mistakes, but most of them do not result in a collision, and most of the ones that result in a collision do not result in a death.

I also think it's inevitable that computers will become better at driving than not just the mean or median driver, but 90th percentile or more, at which point it would be immoral to not, in some way, encourage self-driving cars over human-driven cars. My guess is that we are less than a decade away from that point, but I could be wrong.

Certainly the lack of a safety-culture in the automotive field (as compared to say, avionics) is a huge barrier to be overcome, and that's a social hurdle, not a technical one and I can only make wild-ass guesses as to when that will change.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by perl4ever on Sunday November 10, 2019 @01:53AM (#21496794)
A Tesla owner could ride in a cab or on a bus, though. On average, it seems like without a Tesla they would be at least minimally more likely to.

You seem to be implying that cab drivers are the only people who ride in cabs.