law of unintended consequences. I can even imagine a net increase in accidents until the technology is mature enough to actually take over full responsibility. So far all of the assistive technologies were straightforward to understand and control - by that I mean ABS, ESP, velocity keeping, and autobraking. You were never tempted to think that the car will drive for you with these. But the moment you add lane keeping and take over the steering wheel, there's no immediate obligation to pay attention anymore. It's deep in the uncanny valley and I'll have none of it until we have fully autonomous cars.

Another user linked a Fifth Gear video testing the auto-braking systems in Volvo/Mercedes/VW cars. In it, they say the auto-braking systems only engage by braking hard at the last second specifically so you don't get blasé about it as if it were a smooth controlled braking. It's not a mode you want to use because it really feels like you're going to crash. https://www.youtube.com/watch?v=PzHM6PVTjXo

that's interesting, I can see the reasoning behind, but what happens on a slippery road?

That sounds like exactly what happened in the crash scenario. The driver tapped the break, said "meh, the car will handle it" and then plowed into the car in front of them.

I'm pretty leery of these systems at the moment due to people using them like this. That's just not what they are ready for at the moment and people's unrealistic expectations are leading them down dangerous paths.

Is this not basically what Autopilot is doing for driving in general?

Humans can be trained into not having surprise paralysis, for instance, through a simulation. It can be part of the requirements for driving permits.