News for hackers. Stuff that matters.
Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Monday October 05, 2020 @09:31PM (#24693954)
When the creator of your car makes comments like

"Your hands are just there for regulatory reasons"

Then turns around and blames you for acting like that's true when a known defect kills your wife

(aside: Why do you think Mobileye had the same tech as AP1 in multiple other cars but never turned it on continuously? Why do you think it was always limited to highway speeds and correcting a lack of input? Well that's because it can't tell an overhead sign from a stopped firetruck very well...)

Are we really then allowed to go back and go "well they had a nice disclaimer".

-

It's funny really, because calling it Autopilot wasn't enough (spare me the airplane comparisons). Now there's a feature list called "Full Self Driving... capability"

Like, the term FSD is used as part of a label for a collection of features, of which NOT A SINGLE ONE is even a COMPONENT of self driving, because EVERY SINGLE ONE requires full time driver attention.

They're literally using the fact that theoretically, one day, the SENSORS MIGHT OVERLAP WITH WHAT FSD REQUIRES to sell it as... not FSD. FSD capability.

What a joke.

-

Tesla has gotten away with homicide, and I guess we're ok with them continuing to.

The useless comparisons about "Tesla with AP is safer than Tesla without", show me the comparison that Tesla with AP is safer than AEB+LKA+ACC (pretty much AP, minus the party trick that lulls drivers into a false sense of security), and I'll show you a made up study.

Because that would be AP being exactly what it already is, minus the one mode where the driver is supposed to be a safety net for the computer. Where the driver already has responsibility to make sure it doesn't crash! It's literally like removing the ability to steer your bike with the training wheels, the training wheels are still there!

It's simple really, Tesla has put marketability of a safety feature, over the actual safety benefits, at the cost of human lives.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Tuesday October 06, 2020 @01:10AM (#24694889)
I agree that the marketing is misleading and unreasonable but I think the last part of your comment is going too far.

I drive a car without any of those features. I think my car's level of safety is acceptable. If AP can beat that level of safety, it's good enough. It's okay if AP isn't the absolute safest mode, just like it's okay if companies sell cars that "only" have a 4-star crash rating.

Re:Tesla hacker reveals what driver-facing camera is looking... by theshrike79 on Tuesday October 06, 2020 @01:57AM (#24695079)
Exactly this. People are expecting car "autopilots" to exceed all human drivers in every situation, which is a completely silly place to move the goalpost to.

I see it more as an assistant that makes us all better drivers by letting us concentrate on stuff the computer can't do.

Even my Hyundai can drive automatically perfectly well at highway speeds and follow the road and even slow down with the traffic. This lets me relax behind the wheel and concentrate on other things than keeping the car in the lane.

I feel I'm a lot less tired and more anttentive when driving with an "autopilot", all the little micro-adjustments needed vex on my brain like nothing else.

Re:Tesla hacker reveals what driver-facing camera is looking... by dafoex on Tuesday October 06, 2020 @05:12AM (#24695949)
I agree, but I interpret the argument as such: Tesla, by using language like "your hands are only there for regulatory reasons" makes no effort to dispel the myth to the driver that the car autopilot can exceed all human drivers in every situation. It aught to be on Tesla's head to instruct the driver that they are the safety net to the computer.

To use the aeroplane analogy that Tesla wants to market: the computer is the first officer in charge of flying the plane, the driver is the captain in charge of everything else.

Re:Tesla hacker reveals what driver-facing camera is looking... by theshrike79 on Tuesday October 06, 2020 @06:50AM (#24696410)
Yea, the Tesla marketing team has gone a bit overboard and some people are taking their spiel as gospel.

On the other hand, event the new VW ID.3 can drive on regular roads, manage roundabouts and adjust speed according to speed signs, they're just not making a huge fuss about it.

Re:Tesla hacker reveals what driver-facing camera is looking... by judge2020 on Tuesday October 06, 2020 @08:43AM (#24697075)
Do you have a source for that quote? I searched elon's tweets and Google but couldn't find anything.
Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Tuesday October 06, 2020 @02:02AM (#24695094)
No one is saying AP should be the safest thing ever, my comment certainly isn't saying that AP should be some sorr of super-human driver

The problem is AP is actively making itself less safe than it would be without marketing oriented features even with no additional strengths.

AP could the same collision avoidance and lane centering capabilities, but not allow the driver to engage them in "Autopilot" style. That's exactly what the AP1 hardware did in other cars.

AP would still avoid every accident it could before, but now additionally not kill people when it fails to uphold the very strongly implied (even if legally disclaimed) promises it's marketing and creator love to make.

-

AP has a set of convenience features and safety features.

The convenience features all come with the disclaimer that the human must be attentive at all times while they're engaged.

And in cases where the human is not full engaged because they're placing too much trust in the convenience features, suddenly the convenience features are creating a less safe situation than if the driver did not have them at all, since the driver is now driving distracted.

The safety features cannot close the gap between a distracted driver and a non-distracted driver, that's why they're not seld-driving cars.

-

Or to put it another way, good enough is good enough for the safety features. But "good enough" is actually the most dangerous level of the convenience features, since they lull operators into a false sense of security with deadly consequences.

That's why Google abandoned their equivalent project (which Elon spoke to them about before developing AP. Google's project was called... Autopilot).

It was demonstrated that giving drivers that sort of "good-enough" convenience feature would be encouraging distracted behavior. But that didn't stop Elon.

-

In fact, thinking about it, how sad is that? Google used cameras in their vehicle in conjunction with AP features and proved that people would misuse the system.

Knowing that Elon was in contact with them about the program, isn't it such an odd coincidence that Elon has pushed back on driver monitoring?

It's almost like he knows exactly what it would reveal, but doesn't want to invite the trouble that comes with tracking such an inconvenient truth.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Tuesday October 06, 2020 @02:17AM (#24695169)
> The problem is AP is actively making itself less safe than it would be without marketing oriented features even with no additional strengths.

That's a shame but I'm not going to worry all that much about it killing people if it's still safer than a basic car.

> Or to put it another way, good enough is good enough for the safety features. But "good enough" is actually the most dangerous level of the convenience features, since they lull operators into a false sense of security with deadly consequences.

No. We were talking about the final level of safety after factoring in the hubris-caused danger. The level of safety before factoring in hubris was much better than "good enough".

Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Tuesday October 06, 2020 @10:54AM (#24698269)
> That's a shame but I'm not going to worry all that much about it killing people if it's still safer than a basic car.

This doesn't make sense unless you just like senesless death.

It can be safer than a basic car and not kill people.

Or it can be safer than a basic car and kill people for the sake of marketing.

Since like I said, AP should not leave you feeling rested, full attentiveness is required, more than usual. If you're operating under the assumption you can relax a little with it active like a co-pilot or watcher, you're following for it's most dangerous trap.

-

Also you realize basic cars have been picking up the safety features for a while now right?

The Toyota Corolla, the baseline of basic cars, has had LKA for 3 or 4 years now? And will warn a drowsy driver in case of excessive interventions.

It's sad again, because if anything Tesla is poisoning the well on these features since they're lulling people who don't even have the cars into a false sense of security.

AP marketing makes it seem like it can make a drive more relaxing (with Tesla salespeople using that exact phrasing on a test drive mind you) when the moment it does that, it's making you less safe than if it wouldn't pretend to be capable of that.

Now when other manufacturers intentionally omit continuous lane centering, it seems like an omission of a convenience, not an omission of a dangerously misleading liability to the safety of its users.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Tuesday October 06, 2020 @10:07PM (#24704629)
If the car only saves 20 lives when it could have saved 70 lives, I don't care if you want to call that "killing people", I think that car is fine.

I don't "like senseless deaths" but I think it's acceptable if people like the former car better than the latter car and purchase more of it.

> If you're operating under the assumption you can relax a little with it active like a co-pilot or watcher, you're following for it's most dangerous trap.

If it's still safer than a car without those features even after falling into the trap then my reaction is a big shrug.

Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Wednesday October 07, 2020 @12:22AM (#24705277)
Here the car saves 100000 lives, but kills 5 people, when it could have just not killed the 5 people! And still saved the same 100000 lives!

That's a problem! To any normal person, that's a problem.

You're doing this weird math where actively murdering 5 people but saving 100000 is the same as just saving 99995, but it's not when the 5 murders were not a requirement to save 100000.

Maybe your confusion is thinking the safety features are the ones doing the killing when they're not? It's extra trimmings built on top of the safety features that are doing the killing.

In other words:

> If it's still safer than a car without those features even after falling into the trap then my reaction is a big shrug.

It's not safer! Because that sentence is about the convenience features are not the safety features!

LKA, AEB, Crash avoidance, those all operate separately of the convenience features! And it's not a guess about how the internals work either, Tesla literally sells things like "Navigate on Autopilot" separately.

It's really simple logic. The safety features save lives. The convenience features, by definition, cannot save lives that the safety ones didn't.

When a Tesla avoids hitting a car coming into it's path, it's not because Navigate with Autopilot was on, it's the Active Safety Features function:

https://www.tesla.com/support/autopilot

But when a Tesla hits a stopped firetruck after Autopilot lulled the driver into thinking they wouldn't have to brake, the active safety features can't do anything! Because the convenience features are subject to the same limitations as the active safety features.

---

That's the crux of the issue that you keep missing. No one is saying the active safety features should save more lives, they're saying the convenience features need to stop tricking people into driving distracted. Because those people sometimes die in accidents that wouldn't have happened if they didn't have them.

The safety features would still save the same number of people, just no one would die needless deaths in the name of marketing and looking cool.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Wednesday October 07, 2020 @03:40AM (#24706116)
> You're doing this weird math where actively murdering 5 people but saving 100000 is the same as just saving 99995, but it's not when the 5 murders were not a requirement to save 100000.

I don't see it that way.

It's a completely random set of people that die either way. And all the deaths basically boil down to "driving is dangerous". It's not one set of people being saved and a different one dying. It's all one group, so only the total number matters.

> It's not safer! Because that sentence is about the convenience features are not the safety features!

Your sentence was about that. My counterpoint was about more than that.

I'll reword my point without the word safer: If the risk reduction from the safety features is bigger than the risk increase from the convenience features, I think things are acceptable.

> The safety features save lives. The convenience features, by definition, cannot save lives that the safety ones didn't.

I wasn't trying to say the convenience features saved lives.

> No one is saying the active safety features should save more lives, they're saying the convenience features need to stop tricking people into driving distracted.

I'm the one saying that safety features need to save a certain number of lives. And that number is "enough to make up for the risk caused by the convenience features".

> Because those people sometimes die in accidents that wouldn't have happened if they didn't have them.

Eh, someone that's distracted could easily die in similar accident in a dumb car. To me it's all risk vs. risk. Nobody is getting murdered by the car as some kind of sick trade-off. It's not a trolley problem. It's just undoing some of the safety. But undoing the safety is acceptable, just like a car that had none of those safety features (and none of those convenience features) would also be acceptable.

> The safety features would still save the same number of people, just no one would die needless deaths in the name of marketing and looking cool.

If it was just marketing and looking cool that would be one thing. But it's also quite hard mentally to pay attention when everything is on auto. And it's a much nicer driving experience that's inherently dangerous no matter what the marketing says. So I don't think it's possible to have the convenience without the danger. You'd have to disable AP.

Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Wednesday October 07, 2020 @09:58AM (#24708241)
> It's all one group, so only the total number matters.

You're very alone in this line of thought.

If you save 100000 people _by murdering_ 5 people, it's unfortunate and a hard choice, but at least arguably good. Most people will be ok with this (this is pretty much how the world turns to some degree)

But if you save 100000 lives, then murder 5 victims. And those 5 victims were just murdered to murder tham, no one being saved was resting on it.

Most basically ethical people would consider that indefensible.

Maybe you're confusing this situation with the former, where lives are treated as fungible to save great numbers.

That can be tricky ethically, but at least the greater good is on your side. Here you're treating lives as fungible for the sake of it.

It's not really an admirable thing to lack the ability to see why that's wrong, so I implore you to dig a little deeper.

This isn't sacrificing people to help more people, it's helping people then sacrificing some people for fun.

If you talk someone down from a ledge do you now go around thinking you get to murder someone and it cancels out?

> If it was just marketing and looking cool that would be one thing. But it's also quite hard mentally to pay attention when everything is on auto. And it's a much nicer driving experience that's inherently dangerous no matter what the marketing says. So I don't think it's possible to have the convenience without the danger. You'd have to disable AP

This is literally what every comment in this entire thread by me has said. Maybe you finally get it.

Car without convenience features saves lives doesn't kill.

Car with them saves same lives and kills.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Wednesday October 07, 2020 @05:44PM (#24713109)
I don't think you understood what I was saying.

I basically agree with everything you said about saving some people and murdering others.

But "saving less people" is a totally different situation. If I donate some money to charity, and I'm asked to donate more money, it's okay for me to say no.

So the question is whether these features fall under "saving some people and killing others" or if they fall under "saving less people".

You believe it's the former, and I believe it's the latter.

Remember that even pure safety features are imperfect and they are subject to the butterfly effect. Anything you change about a car design will cause some shift in who dies while driving that car. You have to determine if there are really two separate risk groups, one sacrificed for the other, or if it's all really one risk group.

As a thought experiment, imagine we made a car that was all-around much safer, and then we let 14 year olds drive it. Well that actually happens, with mini-cars that have strict speed limits. (Usually they're legally not cars.) We could only allow normal licensed drivers to use them, and they'd be very safe! But then, purely for convenience, we let younger teenagers use them. Is that a moral abomination? Some of them are going to die. But I think it fits into the normal risk of driving, and as long as a 14 year old in one of those is safer than a normal driver in a normal car, I'm satisfied.

Re:Tesla hacker reveals what driver-facing camera is looking... by BoorishBears on Wednesday October 07, 2020 @05:50PM (#24713163)
> So the question is whether these features fall under "saving some people and killing others" or if they fall under "saving less people".

There are separate sets of features that you can't just glue together, they're not even offered together, you have to pay separately for the second set.

One set saves people, one set kills people. The set that kills people, doesn't save people.

You think it's ok to sell the set that kills people because they have a set thats save people.

I know it's not ok to sell features that kill people and don't save anyone. It'd be one thing if it could save people, but it doesn't. The other set does that.

This set just kills people. Doesn't save anyone.

Stop selling it.

Simple as that.

Re:Tesla hacker reveals what driver-facing camera is looking... by Dylan16807 on Wednesday October 07, 2020 @07:24PM (#24714005)
> You think it's ok to sell the set that kills people because they have a set thats save people.

Only if all three of the following are true:

A) they both affect the same risk factor (Just because they're "separate" features and sold separately doesn't make this false. And the way the autopilot is so intertwined with the others, I don't think it's really fair to say they are truly separate features. They build upon each other.)

B) it's impossible to run the dangerous one without the safety one

C) running both is safer than running neither

If you want to make cars safer, then you should be insisting that all cars be safer. The car that kills people because it lacks any of these electronics is just as bad.

To elaborate on (A), with these particular features I don't see the car as causing distinct people to die compared to who it's saving. It's not trading between different groups of people in any meaningful way. It's just a single number of how many people die per mile. With other features you could get into that moral dilemma. With these features I don't think it does.

Re:Tesla hacker reveals what driver-facing camera is looking... by dafoex on Tuesday October 06, 2020 @05:18AM (#24695983)
>The level of safety before factoring in hubris was much better than "good enough".

I think the point of the argument is "ship the system that doesn't cause hubris if it is much better than 'good enough'"

Re:Tesla hacker reveals what driver-facing camera is looking... by totalZero on Tuesday October 06, 2020 @04:11AM (#24695653)
I don't think it's going too far. There are certain features of a vehicle for which the manufacturer could be forgiven for using borderline deceptive marketing to move the product. Yet "Autopilot" that isn't capable of fully replacing a driver, is marketing doublespeak that has contributed to the deaths of Tesla motorists who ostensibly believed that "Autopilot" actually means autopilot.