News for hackers. Stuff that matters.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by mindfulplay on Sunday November 10, 2019 @12:24AM (#21496537)
Just listening to this talk scares me. The amount of errors - even in a seemingly normal, sunny day - is mind boggling to think people trust this crap.

How can we rely on the output of eight cameras? This is not a kid's science project.

It's all fancy neural networks until someone dies. Pretty callous and Silicon valley-mindset for such an important and critical function of the car.

Will never buy a Tesla after having seen this.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by panarky on Sunday November 10, 2019 @12:53AM (#21496631)
> mind boggling to think people trust this crap

It's also mind boggling to think we currently trust organic tissue to do this crap, some of which is bathed in psychoactive chemicals.

And yet we do, and as a result, horrendous catastrophes occur every minute of every day.

> It's all fancy neural networks until someone dies

No, that can't be the standard, not when people are dying right now in the current regime.

Unless the new regime kills and maims at a higher rate than the current regime, there is no reason for fear.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by thebruce87m on Sunday November 10, 2019 @03:21AM (#21497071)
No, it needs to be a lower rate since it will kill at random. Today’s rate includes drunk drivers, people on their phone and other “unsafe” drivers. If you are an attentive driver your chance of death would actually go up if the overall death rate was the same.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by slim on Sunday November 10, 2019 @11:19AM (#21498587)
if you are an attentive driver you can get killed anytime by these other drivers
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by thebruce87m on Sunday November 10, 2019 @11:37AM (#21498683)
I’m maybe missing your point, but this is reflected in the overall death rate now. Similarly in the future the overall death rate will include deaths of the occupants of self-driving cars and also the cars/people that they hit.

The perception will be different however, since in the public eye the drunk driver “had it coming” and the only real loss is the other driver/passengers/pedestrians. If a self-driving car kills its own occupants and the occupants of another car then there are more “innocents” than in the drunk-drive scenario.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by marvin on Sunday November 10, 2019 @09:15AM (#21498018)
Thankfully, using these features is optional.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by semi-extrinsic on Sunday November 10, 2019 @04:19AM (#21497223)
To really be a measurable improvement over humans, and especially an improvement that is statistically distinguishable from just letting the safety tech of 2019 percolate into the average car, self-driving needs to achieve fatality rates around 0.1 per billion miles.

Current averages for the US are around 10 per billion miles and decreasing; best countries in Europe are already below 5 per billion miles.

There's some really interesting progress both on vehicles monitoring driver attention, and on monitoring for alcohol in the air, that would yield substantial improvements even beyond 2019 tech. I have no doubt we'll see fatality rates below 1 per billion miles in Western Europe within a decade.

The corollary of the statistics quoted above is that you need to observe your self-driving vehicle system over tens of billions of miles before you even know if you're safer or not.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by tim333 on Sunday November 10, 2019 @06:23AM (#21497543)
I'd quibble with your stats that self driving has to be 100x better than human to be a measurable improvement. You can estimate fairly well if a human driver is safe or not from a few hours as a passenger by seeing if they notice everything and if they have near misses. Also it's not to hard on published data to see the Tesla autopilot seems a fair bit worse than humans under the same conditions. I imagine they will improve.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by semi-extrinsic on Sunday November 10, 2019 @03:51PM (#21500122)
I'm not saying it has to be 100x better than a human.

What I'm saying is, the averages today are across all vehicles on the road, some of which are old and have fairly low survival probability in an accident. And the deployment of technology to handle/avoid distracted driving is still in its infancy. If we just wait 10-15 years, I believe the safety rates for humans will increase by 10x over current US statistics.

Then self-driving has to manage another 10x improvement over that, in order to be worth it.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by perl4ever on Sunday November 10, 2019 @01:32AM (#21496741)
"It's also mind boggling to think we currently trust organic tissue to do this crap, some of which is bathed in psychoactive chemicals."

The appropriate standard for self-driving cars is a sober professional driver, not the average idiot on Saturday night. It isn't unusual for a driver to go a million miles without an accident, when that's their job.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ajross on Sunday November 10, 2019 @01:41AM (#21496759)
No, the appropriate standard for a self-driving car is the driver they replace. If they're safer than whoever would have been driving (be it on a saturday night or not) then they're a net win. Right now the average idiots are buying more Teslas than mail carriers and cab drivers, though that might change I guess.

Numbers are numbers.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ars on Sunday November 10, 2019 @02:00AM (#21496820)
If you like numbers, then you'll like this one: Humans achieve 7 9's of reliability when driving a car, as measured for fatalities by time, or 8 9's if you measure by miles.

I've yet to see a computer just stay up that long, never mind actually doing anything the whole time.

(If you use the collision rate instead, the numbers are about 500 times worse - but that's still quite a lot of 9's.)

People have this fixed thought that humans are terrible drives. They are not! Computers have their work cut out for them to exceed the reliability record humans have set.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by aidenn0 on Sunday November 10, 2019 @02:09AM (#21496847)
You are using the 9s term wrong. It is supposed to be used with unitless numbers. Not crashes per hour or per mile.

Silly example of why: humans are zero nines of reliability if you talk about crashes per parsec.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by ars on Sunday November 10, 2019 @02:19AM (#21496879)
> Not crashes per hour or per mile.

But that's not what I measured. I measured safe minutes of driving as a ratio to unsafe minutes. Which is a unitless number.

For 7 9's I assumed a crash had a 5 minute lead-in of unsafe driving before the actual crash, and that average driving speed was 30 mph.

If you assume the bad driving is 30 seconds (for example the accident in Tuscon the Uber car saw the pedestrian around 30 seconds before crashing), then you can add another 9, making both figures 8 9's of reliability.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by aidenn0 on Monday November 11, 2019 @01:53AM (#21502757)
Thanks for the clarification. I haven't checked your numbers but they seem reasonable.

FWIW, I think deaths, regardless of fault, is probably the best number for an apples-to-apples comparison. If the Uber car were driven by a person without a dash-cam, then there is no way they would have been ruled at fault.

Similarly, I have been involved in 5 collisions (most were not my fault). Of the 5, 3 were reported to insurance and only 1 was recorded by the police (For 1 other the police were called, but the dispatcher (on the non-emergency police line) said "Are both cars driveable? (yes) Is anyone injured? (No), then don't bother us!" (By comparison, in California even a single car collision on private property must, in theory, be reported to the police).

Deaths, at least, will get recorded by the CDC, if nobody else.

[edit]

However, I don't think that deaths are necessarily the best number for considering "failures" all deaths are in some way a failure, but most failures do not result in deaths. Nobody even had to see a doctor for the 5 collisions I mention. I've fallen asleep at the wheel and crossed the center-line without getting in an accident, &c. humans make a lot of mistakes, but most of them do not result in a collision, and most of the ones that result in a collision do not result in a death.

I also think it's inevitable that computers will become better at driving than not just the mean or median driver, but 90th percentile or more, at which point it would be immoral to not, in some way, encourage self-driving cars over human-driven cars. My guess is that we are less than a decade away from that point, but I could be wrong.

Certainly the lack of a safety-culture in the automotive field (as compared to say, avionics) is a huge barrier to be overcome, and that's a social hurdle, not a technical one and I can only make wild-ass guesses as to when that will change.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by perl4ever on Sunday November 10, 2019 @01:53AM (#21496794)
A Tesla owner could ride in a cab or on a bus, though. On average, it seems like without a Tesla they would be at least minimally more likely to.

You seem to be implying that cab drivers are the only people who ride in cabs.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by freediver on Sunday November 10, 2019 @12:40AM (#21496592)
You are right this approach is scary, and it is astonishingly innacurate (im a tesla owner).

However the reason are not eight cameras. You should be able to drive fine with just one camera (thought experiment: could you drive a car 1000 miles from you, just by seeing what the driver of that car would see, no extra cameras, sensors or lidars?).

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by mdorazio on Sunday November 10, 2019 @12:50AM (#21496621)
One stereoscopic ultra-HDR 4K camera would be fine... if it was backed by a strong AI. To even suggest that ML is anywhere wen remotely close to this level is the height of hubris.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by freediver on Sunday November 10, 2019 @09:34AM (#21498066)
Not even stereoscopic, you could still drive with one eye shut.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by mindfulplay on Sunday November 10, 2019 @12:47AM (#21496608)
I work closely on ondevice OCR technology - something that has improved so much and still cannot figure out the difference between I and 1.

And that took many decades not even years and in a very constrained problem space.

Releasing a "self driving" metal torpedo even if it's a limited Summons should at least be regulated if not outright banned.

I feel like even a single accident let alone death should put these idiots behind bars.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by rawoke083600 on Sunday November 10, 2019 @04:19AM (#21497224)
How do we currently regulate and put behind bars the human drivers that drives the metal torpedoes like idiots ? Specifically the ones that are not caught.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by jayd16 on Sunday November 10, 2019 @12:51AM (#21496625)
It is regulated. Would you like it to be regulated in a specific, different way?
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by optimiz3 on Sunday November 10, 2019 @12:32AM (#21496567)
Your comments read like /r/RealTesla | TSLAQ fud.

Please explain, in detail, what your specific objections are, and how you are more qualified on this subject matter than the presenter.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by mindfulplay on Sunday November 10, 2019 @12:44AM (#21496601)
I am honestly scared even of WayMo that I think unless NHTSA gets its acts together this shit shouldn't be allowed on roads.

It's often reactionary - people wait for someone to die and then suddenly you have nervous Musk in front of Congress and such crap. Why wait? Why can't these be regulated before allowing on roads?

And no, failover control is not acceptable given the past incidents and deaths.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by Joky on Sunday November 10, 2019 @01:10AM (#21496686)
You seem to use "regulated" as in "forbidden" right now, but maybe I am misreading so can you elaborate?

Today thousands are dying every year on the road, are we just forbidding cars entirely?

> And no, failover control is not acceptable given the past incidents and deaths.

How many incidents/deaths per miles driven? How does it compare to all the other transportation systems?

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by perl4ever on Sunday November 10, 2019 @01:39AM (#21496753)
Why are people so committed to the idea that self-driving cars are anywhere near human standards? It just seems like a groundless assertion of faith to me.

Professional drivers can go for a million miles without an accident, and I don't believe anyone's autonomous driving software can get within an order of magnitude of that without a disengagement, even in favorable conditions.

You may disagree that a disengagement is equivalent to a human having an accident, but I strongly feel that it is. In either case, you have a situation where the driver reached a point where it was definitively unable to determine an appropriate next action.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by tim333 on Sunday November 10, 2019 @06:34AM (#21497570)
I don't think most people think they are near human standards now. But there is the prospect for them being there before too long.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by JohnJamesRambo on Sunday November 10, 2019 @01:31AM (#21496740)
https://www.greencarreports.com/news/1119936_tesla-fatal-cra...

“Perhaps that’s because, as it turns out, Teslas on Autopilot could have a higher fatal accident rate than those driven entirely by humans.”

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by chrisco255 on Sunday November 10, 2019 @01:14AM (#21496696)
The only way to train these things is on actual roads. As far as I know, all of these systems require a human driver present at this time. None are fully autonomous.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by tim333 on Sunday November 10, 2019 @06:38AM (#21497578)
Waymo have done some very limited full autonomous https://news.ycombinator.com/item?id=21427036

They still have backup drivers mostly though.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by natch on Sunday November 10, 2019 @01:46AM (#21496775)
It does require human supervision while driving.

Tesla makes this very clear.

It's also all over the Internet.

But there will always be people who do not get this.

Those people should not drive Teslas, or pretty much any modern car for that matter.

If you are under the impression that you would be relying on this system to drive the car, then I agree you should not get a Tesla.

Of course full self driving is coming at some point, but that's a conversation for another day. Meantime Tesla is making steps toward it very incrementally with things like the "stop mode" rolling out right now.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by yo-scot-99 on Sunday November 10, 2019 @01:03AM (#21496665)
this tech is interesting but so poorly understood that it's just using the (public) roads as one large alpha test. given a NN there is no way to verify what safety ranges are there. for instance if each camera slightly changed exposure or occlusion are the results smoothly changing? all they can do is try it and hope the inputs are in a safe part of their optimization space.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by 0-_-0 on Sunday November 10, 2019 @05:40AM (#21497428)
Just replace "eight" with "two" and this could have been written about the human brain
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by oska on Sunday November 10, 2019 @05:54AM (#21497472)
And driving with only one is still legal.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by anticensor on Sunday November 10, 2019 @01:29PM (#21499354)
Monocular or deaf driving is subject to many restrictions including being unable to drive a commercial vehicle.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by matz1 on Sunday November 10, 2019 @02:02AM (#21496825)
How many actually dies? Do you have the statistic?
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by k2xl on Sunday November 10, 2019 @12:54AM (#21496636)
Hmm how do you explain that data shows that there are likely less accidents and less road fatalities when autopilot is engaged?
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by lsh on Sunday November 10, 2019 @12:55AM (#21496639)
citation, please
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by kjksf on Sunday November 10, 2019 @01:07AM (#21496673)
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by simion314 on Sunday November 10, 2019 @02:15AM (#21496868)
The autopilot is a drive assist , I can explain that there are less accidents when drive assists is one because

- the driver is still present and will most of the time intervene and save his life(there are some youtube videos where the Tesla AI was trying to kill the driver and those incidents don't appear as accidents int the stats)

- "autopilot" is engaged mostly on highways , but the statistics are not accounting for this

- they are also comparing the safety of a new car versus the median of all cars(old and new, cheap or expensive) on all demographics(how many teens own a Tesla?)

I would like that Tesla make all the data open or have an independent group analyze it, including the disengagements, I wonder how isolated are the cases where the car is driving you into trucks or stone wall but the drivers intervened and saved themselves and also saved Tesla from a bad statistic.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by FireBeyond on Sunday November 10, 2019 @02:51PM (#21499788)
Which is entirely cherry picked and optimistic.

Autopilot has the luxury of being able to / requiring disengagement in less-than-favorable driving conditions. Humans don't.

A fair comparison would be comparing against the miles/conditions/roads driven.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by newnewpdro on Sunday November 10, 2019 @12:32AM (#21496572)
Same.

Watching the AI visualization of summons in-action was horrifying, and made clear why many have reported summons mode as resembling a drunk person navigating a parking lot.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by natch on Sunday November 10, 2019 @02:12AM (#21496856)
Yes it's not perfected yet, which is why it requires human supervision for now. Having it operate in the wild as it is now (again, under human supervision) will help it become less horrifying, which I think you would agree is what we want.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by newnewpdro on Sunday November 10, 2019 @03:50AM (#21497149)
I don't agree, as I don't particularly value/want a future with self-driving vehicles.

I also think the way Tesla is going about this is utterly idiotic and reckless. It upsets me that I'm sharing roads with vehicles having these dysfunctional immature systems.

Point this crap at video game engines and don't let it anywhere near real people until it can drive millions of virtual miles in something like GTA without hitting anyone/anything and without behaving like a drunk driver who lost their glasses.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by natch on Sunday November 10, 2019 @11:23AM (#21498605)
>I don't agree, as I don't particularly value/want a future with self-driving vehicles.

If bad human drivers killed only themselves, that would be one thing, plenty of Darwin awards to go around. But they don't. I do value reducing traffic fatalities to near zero. I think that would be fantastic.

The GTA idea is a good one. I think they've done that, many times over, but I could be wrong.

They way Tesla is doing it is interesting. Have human supervision, so even though the system may behave as you say like a drunk driver who lost their glasses, at least the human is there keeping it in check.

Humans aren't perfect, but I've found (and statistics have shown) that the system plus a human is better than a human alone. That alone should make you less upset. These cars are less reckless than cars driven by humans.

Human supervision is just one element of the very smart way Tesla is going about this. The other element is incremental changes. A lot can be said about that but I won't now. Teslas are getting small and big safety improvements that roll out over time as full self driving gets closer and closer. This should make you feel better.

If you see a Tesla in autopilot, it won't be tailgating. If you see people tailgating in a Tesla, you can be sure autopilot is not turned on. Even with the follow distance set to the lowest, the distance kept is larger than what is done by most humans. This should also make you less upset.

When driving in stop and go traffic with Teslas with autopilot on, if the owner sets the follow distance to say the middle range, you'll find you are often able to merge in in front of them, because they allow space for that (and for safety). This should help too.

Over time, the system is getting better and better. Eventually, it will all be ok!