(a) Even though the driver was getting a warning to put his hands on the wheel, the car should NOT continue driving forward and piloting itself at all with no driver response and insufficient certainty to continue navigating itself.

(b) There was a very obvious hard barrier right in front of the car. Why didn't the collision avoidance detect it and halt the car?

(c) The car was clearly "pulled" to the left by the bright exit lane stripe and missing highway-side stripe.

Simply following stripes seems an absurdly primitive and unreliable technique to me. The fact that the car could be "distracted" this way suggests to me that it isn't getting (or making good use of) a lot of information that should be available to it-- like the behavior of other traffic, the presence of other obstacles and roadside geometry (like the ignored barrier, which was on a collision path), the trajectory on previous trips (and any conclusions drawn then under better conditions), the road sign which indicated a fork (and which was also itself on a direct collision path), the crosswise striping in between the two forks which the car was driving over, and the striping on the other edge of the lane, and the lanes beyond, which followed the correct trajectory.

Each of those things is a significant clue as to the real path of the lane, and are all machine-detectable and processable pieces of information. Even with nontrivial uncertainty about each of them, the totality of information, IMO, should be more than enough to clearly discern the correct lane, especially if efficiently used-- Kalman filter-like information synthesis can take a lot of very uncertain information and synthesize a very certain picture.

Instead the machine ignored the holistic picture and "target fixated" on the stripe. This apparently has a disappointingly long way to go before it is a reliable system.

That seems pretty damning.

On typical sports of city streets, they are much better at detecting stationary objects.

Do you have a link about the radars that are actually installed on this cars can and can't do?

Yes. You know why? Because they can. That's why...

https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...

There are just "assistants", and they can't and should not drive on their own.

They also seemingly purposely make it very vague as to what is actually available when you pay for it today (nothing), what might be available during your lifetime (it will try to kill you) and when it might be actually ready (never)... /s

The FSD option doesn't actually do anything right now or even add any extra hardware.

According to their Q4'17 update letter,[0] "In October 2016, we began equipping all Tesla vehicles with hardware needed for full self-driving capability, including cameras that provide 360 degree visibility, updated ultrasonic sensors for object detection, a forward-facing radar with enhanced processing, and a powerful new onboard computer." All new Teslas manufactured since then include the same Hardware II (or the incremental "II.5" version), which includes the eight cameras referenced under the "Full Self-Driving Capability" option in their configurator. Enhanced AutoPilot activates certain software functionality (what people generally understand to be "AutoPilot"), while the "Full Self-Driving Capability" package doesn't actually do anything right now. You're just purchasing the right to unlock a future FSD update for a $1,000 discount today that may or may not come out before you sell the car.

Basically, Tesla is betting that they'll be able to develop and deploy FSD in a reasonable (for consumers; from a developer and regulatory standpoint, it's incredibly aggressive) timeframe using the hardware already present on its cars. In the interim, FSD orders just sit on Tesla's books as deferred revenue because "FSD" hasn't actually been delivered yet.

As an aside, it's actually kind of impressive that ~40% of buyers[1] have opted for the FSD option despite the fact that it doesn't exist yet. Offering the option today involves some pretty significant risk if there are significant delays or Tesla runs into unexpected development hurdles it can't easily get around. It could easily turn into a class action nightmare.

0. http://ir.tesla.com/secfiling.cfm?filingID=1564590-18-2956

1. https://www.inc.com/kevin-j-ryan/35000-people-have-paid-for-...

But it sure does make Tesla look progressive alright. And upon a cursory, or even second reading, the option does certainly read like you're getting something. That's probably where those 40% come from. Basically, this is a behavior I would expect from some shady operation on Kickstarter, not from a major corporation...

At some point there really needs to be a legal discussion about whether Tesla's argument that the driver must always be alert and responsible for the car is a realistic standard in which to hold people when semi-autonomous systems are engaged.

It seems completely predictable that it wouldn't be, especially when they call it "autopilot."

>To be fair this is mostly a problem with the general public's understanding of the term autopilot and not with the actual systems that have used that name before. Aeronautical autopilots would probably only be classified as level 1 or maybe level 2 [autonomy] when that term was first coined and used to describe them. Even now I am not sure if we would classify the average autopilots you find on a commercial airliner as being fully at level 4.

I don't think Tesla's naming it autopilot should have any impact on any legal discussions. I am not sure the driver should be responsible for everything that happens to the car while autopilot is engaged. However I think it is a completely fair and reasonable expectation that a driver understands what autopilot is capable of doing before they engage it. I don't think Tesla (at least through their marketing, I have heard some questionable things about their salespeople) has been misleading about that in any way. I don't have a lot of faith in people handling a Tesla responsibly if the only thing they know about autopilot is that it is named autopilot.

Personally I think Tesla should bear some responsibility for the name. People don't know what aeronautical autopilots do and don't do. Laypeople think autopilot does the driving for you.

Pilots can do that because there are no other planes in the sky on their route because some outside human is taking care of that.

This should have been done before these things were ever allowed to put on roads.

Zuck is right. We naively trust the world at large to work for our benefit automatically. We truly are "dumb fucks".

Edit: it's in Chicago not Indiana, see replies

This occurring in multiple locations isn't a comfort, it means the issue is more widespread than a single location.

Compare some from the UK.

Bumpy paint, so the wheels vibrate and so they show up in the wet. Red reflectors, and barriers that would push the car back onto the road rather than into the bridge support.

https://goo.gl/maps/TUf1xzLUN5N2

This is normal, though they're aren't many roads like this within cities, so there's usually space for a long, grassy strip.

Maybe you aren't used to concentrating while driving? (I'm only half joking, an American friend one made a similar remark when he visited me.)

https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r... (I ignored microstates when counting.)

I can really see why Tesla would try to deny this so hard, seeing it happen yourself is frightening.

I've read both of their press releases on the incident [1] [2] and I didn't see any attempts at denial in them.

[1] https://www.tesla.com/es_ES/blog/what-we-know-about-last-wee...

[2] https://www.tesla.com/es_ES/blog/update-last-week%E2%80%99s-...

It seems like Tesla is trying to explain-away that particular incident as a one-off statistical outlier, and not the fault of the autopilot system. Appealing to statistics like that seems like an attempt to deny responsibility to me.

But unless the person doing the video faked it, there's no question what happened: the car is driving down I-94 and veers out of its lane towards a concrete median until it's stopped by the operator.

Anything less would be denial and warrant some head honcho at some regulator with teeth, and probably the FBI, turn up at Musk's house and drag him away.

From your second link, in Tesla's own words:

"The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken."

What they didn't write is that the car also had an unobstructed view of the crash attenuator.

Stop calling Autopilot. It's adaptive cruise control, so call it that.

I'd also be curious about the radar absorbsion of that rubber impact barrier.

This, in part, is why Musk's assertion that this hardware would be sufficient for level 4 autonomy was so ridiculous. It really wasn't the kind of lanuage that a responsible engineer would use.

And I think that given Tesla's known propensity of logging _everything_, they would ave video footage of those camera's saved somewhere. I wonder if this is submitted as evidence, so we can understand what the car "saw" and "understood", both from optical as well as non-optical sensors...

Also, the warning to put hands on the wheel was a joke. It's incredibly subtle and would only be noticed by someone looking for it, which is probably not the people that most need to respond to the warning.

You can see from the dashboard icons that he disengaged the Autopilot (by taking control). If it had detected an imminent crash it would have made warning sounds.

(I don’t use my phone while driving, so my experience is based on looking at the road. Maybe it’s easier to miss when looking downward rather than forward.)

Can't wait to see Tesla close the issue in Jira with a "Cannot reproduce."

That said, someone purposefully doing this and stopping in the middle of the highway stupidly dangerous.

That said, someone purposefully doing this and stopping in the middle of the highway stupidly dangerous.

As the saying goes, "it had to be done" --- to show the defect, before someone else gets killed, and the driver was clearly paying attention here. It stopped pointed right at the divider, a position no car would normally be in anyway, and there weren't many other cars around.

That said, self-driving cars have to be able to handle poor markings like this.

But what is equally damning is that I can't tell that the Tesla has done anything wrong until it's moments away from hitting that barrier. Who the hell decided that was a sensible way to mark the road?

I would strongly prefer no markings to that confidence-inducing single white line, which I have been trained to through years of driving to stay sensibly adjacent to, and which will lead me like a lemming to my death.

At 19 seconds, you can see that the dashboard says to put hands on the wheel. Does that not trigger an audible alarm by default in a Tesla?

No. That's a standard warning that triggers every few minutes if the car can't sense your hands on the wheel. If you don't place your hands on the wheel at that point, then there will be an audible alarm. If you ignore that, the car will slowly reduce speed and come to a halt.

*every few seconds (12 seconds, I believe)

It probably would've been fine if there was another white line at the split, like this (marked in red here):

https://files.catbox.moe/10p2hh.JPG

That's not an unusual road condition at all; also: you'd think there'd be some logic reconciling the line it thinks it has to follow with the fact that an entire lane of traffic on the Ryan (the left-most I-94 lane) doesn't abruptly end.

This does bring up a theory: the lane keeping is biased towards holding itself to the left edge of lanes, on the basis that most exits are right exits and therefore the right edge will tend to diverge a ways before making a new lane. I'm curious how it would handle the split at the mixing bowl in Springfield, VA: https://www.google.com/maps/@38.7735345,-77.1816237,3a,19.8y... (functionally a left exit, although signed as if it were a right exit). The solid white line doesn't actually denote any lane exit boundaries there...

Waymo is on a whole other level.

But I really would love to see more collaboration rather than competition in this area where safety is critical. Pardon the pun - why does everyone need to reinvent the wheel? And also encumber each other with patents?

Open source beats proprietary competitive stuff like this.

Is Waymo really "there"? It seems like everywhere I turn I'm seeing things that suggest that all of this technology has been hyped far beyond its actual capabilities.

For Uber this number was 13

5600 miles starts becoming "I need to use the wheel only 4 times a year" territory.

It sounds like you're saying I will very likely die 4 times a year.

If anything, perhaps the better performing algorithms will lead to more complacency (inattention) on the part of the driver...

Sort of a Catch-22? (Or is there a better analogy I'm not thinking of.)

Call me a Luddite, but I think I would prefer to steer the car myself, thank you. When someone says, self-driving car, I assume they mean a taxi, bus or train: all well-proven technologies.

I agree that this isn't enough for full autonomy. And that partial autonomy can lead to false security. But I bet there are a lot of drivers who have more than 1 incident every 5k miles.