The degree of care that went into the e-stop design of the lathe and mill controller and the number of checks and tests of the fuel estimation software are stuff that I'm (hopefully justified) proud of, that's not stuff to cut any corners with. In both cases that took me longer than Hotz claims to have spent on creating the whole self driving car code, there is no way you can do this safely with that mentality. It is absolutely irresponsible to take that attitude towards systems like this, life is not a video game with 3 more lives if you lose this one (or cause others to lose theirs).

For a machine with the potential to easily maim and kill, to be completely autonomously operated in a chaotic environment where it interacts not just with other computer controlled machines but also with humans, even children, I honestly hope that the amount of safety precautions required will grow very quickly.

I like Tesla's and what Elon's accomplished has been impressive but he seems hellbent on "winning the race" to release a self driving car and that worries me.

Almost.

Alternatively, earlier editions of Model S shipped without autopilot, which was then sold as an aftermarket upgrade as late as this spring (when autopilot was an optional $2,500 package on a new Model S).

A car is a bit more complicated to leave stuff out I am sure and the volumes far different. It would likely require at least more resources to have separate parts of lines for some of that as well as different check procedures as so on. But like you said, paying more could help offset that, though it's a gamble for the manufacturer.

I'm more in the camp of please do not release anything driving a car until you guys can figure out how to secure basic software, keep websites running, parse input properly, and so on. In other words, I'd rather them wait decades if that's what it takes.

Personally, I'd rather be killed by a person rather than a person's dumb choice trusting technology. What concerns me in all these situations is that I did not at any point agree or vote to allow someone in a self-driving car to drive on the same road as me. Until they can mathematically prove 100% it is safe in all conditions and situations, I'm not buying things like "driver assist" if a big part of the justification is that AI is better than a human driver (if so, then why assist?). Others may disagree, so be it - I'm a weird guy. I also hate cruise control. And fat-free ice cream.

It took airplanes a long time to become safe (through a long and scary process), and now they're safer than my front porch. Whatever the outcome, self driving cars will very likely be safer than human drivers, and that means more lives are saved than lost, even with the occasional horrific problem. It is entirely plausible that self driving cars are already safer than human ones.

Is this a bad thing? Is it not justified by utilitarianism?

This is a perfect example of the biggest and IMO most compelling critique of utilitarianism -- namely, that it's too susceptible to bullshitting.

People decide what they want to do, then completely invent (or, in more sophisticated cases, carefully fabricate) numbers and time-frames in order to hide their unethical opportunism behind the facade of objective moral assessment.

This is especially problematic when you have to estimate things that are pretty much impossible to estimate without completely bullshitting. For example, "number of people killed by not following certain safety standards", or "impact on time-to-completing of some particular safety standard". We're talking about a novel technology, so there's no way anyone has even remotely high confidence in a sufficiently bounded estimate of these numbers. It's all just bullshitting.

> Is thus a bad thing? Is it not justified by utilitarianism?

These are two VERY different questions.

Moreover I don't see how you can conclude on a whim what's possible to estimate and what is not. Given these things take expertise in the relevant field and more than just casual analysis.

Finally, it's not like there is a small margin of error. Can you even really comprehend 30,000 deaths every single year caused by conventional cars and drivers? This is fricking 9/11 times ten happening every year, and you're not even willing to consider that speeding up r&d might save a few lives?

Yes, exactly. Like in so many cases, coming up with good values for the parameters is often the most difficult part of the analysis... that was exactly my point!

> Given these things take expertise in the relevant field and more than just casual analysis.

Except in this case, there are a lot of experts, and they all seem to be taking roughly the same approach.

> Finally, it's not like there is a small margin of error

It's not clear to me, at this point, what your assumptions even are. What percentage of those deaths would be prevented by self-driving technology? How many new types of accidents would be caused by self-driving technology? By what percentage would that number decrease if the regulatory environment were more friendly? And what's your basis for these numbers?

Also, it's perhaps telling that there really hasn't been a single example of regulators preventing self-driving up until this one, which is rather extreme. Google, Uber, and the scattering of startups and car companies have all taken a more cautious road to deployment of entirely their own making.

> and you're not even willing to consider that speeding up r&d might save a few lives?

There's a big difference between speeding up r&d and rushing unproven tech to market.

I don't think the case has been made yet that today's tech will reduce the number of road deaths. I don't think it's unreasonable to be skeptical of the claim "this works great, trust me".

See, we already have cars, it's not like we're going from 'horses and buggies' to cars, it is going from one good solution to an even better one and even though there is a risk at being stuck at the present local maximum for a little longer going down in order to find the peak of this particular mount improbable would not be an acceptable path for the majority of the public.

And without having a time machine or a functional crystal ball you won't know if it is 10K, 100K or a few million people that get killed before you get it right so the bar has been set at 'it needs to be better than the present level before it is acceptable'.

If you know that after ten thousand deaths, the technology will be good enough to start saving more lives every year, then your utilitarian argument works. The problem is that you can't possibly ever know that in advance. It's equally plausible that the last 0.001% will take decades to solve, or even remain out of reach indefinitely. Then you just threw away ten thousand lives for nothing.

This is why the burden of proof needs to be on the new system, and why the precautionary principle applies.

It's like how IUDs (birth control device) are disproportionately unpopular in the United States despite many benefits - all because of one particular brand that caused infections with horrible repercussions:

http://www.motherjones.com/blue-marble/2012/09/why-are-iuds-...

What is the acceptable casualty rate to get from where we are now to where you think we will be in the future. Is your family an acceptable casualty? If not, then why is mine?

Continuing on with the status quo is just as much of an action and a choice as pushing forward with automation. Both choices are going to get people killed. We can either evaluate the choices and choose the one that seems better, or we can ignore the question and possibly have more people die than is necessary, but there's no option where your family isn't at risk.

So do I really want untested first gen software released without extreme safety standards and heavy testing? No. Do I think the net result of that happening would be a safer driving experience? No. Give me the iOS 9 or so of self driving car software or the unix. Not the Amazon Fire phone.

I'm not sure if it's entirely relevant here, since the comment above presumes that these systems are safer than humans, and the reply doesn't seem to be questioning that.

I think it's very likely that these systems will rapidly become far better than human drivers, and probably already are in the domains where they're meant to be used. But we definitely can't just take that on faith and hope it's being done properly.

Is that reduction a result of self driving or advances in other safety (airbags, seat-belts, crumple zones)?

It is clear that your mind is already made up and that you are a self driving car evangelist. Please endeavor to approach these debates with an open mind and in good faith.

As for your question, you basically destroyed the conversation when you asked "Is your family an acceptable casualty?" If you just wanted numbers, you shouldn't have made it personal like that. That's all my point is here.

This is a really fair point but I also don't think it's a bad thing to say this. I think people distance themselves from possible problems by calling things a numbers game but they don't factor in the fact that they could be part of those numbers.

If I say that 27000 people per year die from listening to Mambo Number 5 then I say that we can lower that to 15 if we remix it to take out the deadly frequency but we will have to test it on 20 people to find which is the deadly frequency for sure. That sounds great. A couple people might die but what's that to save 27000/year?

Well, take that and say we will have to test it on 20 people and 5 of those will be your family. That makes the decision a lot harder. Logically it's exactly the same but emotionally instead of "people" it's 15 "people" and 5 of your family.

So, it's important to mention it could be your family because if someone was alright with it because they are looking at the pure numbers, they should be brought back to the reality that we all actually live in a little.

I guess this all boils down to what you think of the Trolley Problem. I'm one of those who say, yes, of course you pull the lever if you have no way to save everybody, it's obviously better.

Of course, but public perception is, you know... not largely logic-oriented. Whether or not you personally find @abduhl's argument convincing, there's still that emotional hurdle you'd have to clear to convince significant numbers of people to sign up.

In the future, 1.2m people die a year so say self driving reduces that to 0.2m/year, that's a million lives saved per year.

You family is thousands of times more likely to die in a conventional motor accident than due to a self driving vehicle. If unnecessary red tape slows things down and causes 100k extra deaths is that ok? Even if someone you know is amongst the 100k?

That said there's an argument system like Hotz's should be tested first. That's probably not going to slow things that much.

That's an imaginary scenario. You have no idea if that's the way it plays out. Let me give you another scenario: the first self-driving cars kill a bunch of people, and no one wants to touch them for years after they finally become safe; delaying the adoption of cars, and causing millions of additional people to die.