This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.

An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.

> This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.

There are already places that regularly confuse human drivers, and those do get fixed (though usually only after someone dies). I don't see why we couldn't do the same for driving robots, especially when they get more common over time.

This is the first car I've owned with "modern" driver aids and it is true that they are somewhat temperamental. For example, the auto full-beam headlights get very confused in fog (which can be dangerous!), whilst I often get a warning noise when approaching sharp bends at a perfectly safe speed.

I spent half an hour outside in the cold at 2am with a borrowed Nissan Leaf. It only made a loud beeping noise when I tried to lock it. Apparently, this noise means that the trunk isn't properly closed and that it therefore won't lock the car for you.

Honest curiosity, but as someone who drives quite a lot (A, B & C licenses) I don't understand why people need help staying in their lane. Flippantly I'd suggest that piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving, though obviously the prevalence of this feature would suggest otherwise.

Any thoughts gratefully appreciated.

I agree! In this case it was novel to see how it worked (because the facility was there and new to us) but I'm more keen on using my eyes and mirrors. I could see how it could be useful in dense fog though, given many drivers don't bother with lights...

I don't believe we'll solve autonomous driving too soon without adapting the infrastructure to it rather than the other way around. At this rate we'll have truly self driving cars (not "straight line, follow the highway lane self driving") right around the time everyone starts taking most personal cars off the road to replace them with something more efficient that better suits cities of millions. But right now too many people and manufacturers are lying to themselves that self driving cars are right around the corner, and even that they're mostly here.

The solution would be to fix the disinformation surrounding self driving tech and fining the hell out of any manufacturer that doesn't make it painfully clear on every channel that the tech is very limited, mentioning also the limits of the system (e.g. types of traffic it was tested on, how many road signs it can recognize out of the total, how much of the regular driving experience do those conditions cover, how well did it do compared to an average human driver). All in "human" language.

As someone who only sells products using ML, I find the discussion here inorganic.

Maybe most of the blame.

But consider this: If a human had the same problem, who would be blamed? If several humans, then the road would be changed, just like this. The process seems sound, but maybe the changes that make it more human-friendly make it more machine-hostile.

New thinking is needed on how we retrain machines after the change, as it necessitates uniformity - the machine would benefit with fewer variations in cattle grids for example. I reckon a simple change to the road markings for all cattle grids would be enough to make them logically identical - be it paint or signs or some other physical indicator.

Though I don't think fully autonomous driving will be possible in near future. There are too many edge cases.

The owner of the vehicle is also 100% responsible for making sure the vehicle is road-worthy. Decertifying the autonomous-driving feature should probably trigger a manufacturer recall, so the features can be disabled permanently (not just a via software update).

Or alternatively, most countries have a periodic vehicle check (like the MOT test in the UK), and having non-certified features enabled could be a reason for failing the test.

I don't know that this is true in this case. It seems like the problem here is the auto-braking feature kicking on, perhaps disorienting the driver or causing a following car to need to panic brake+swerve. This isn't an instance of a driver becoming inattentive because they expect the computer to act for them. Rather it's an ever-present system erroneously deciding to take control of the car - analagous to a mechanical failure. What could a driver do to avoid this behavior, besides knowing to disable that system ahead of the hazard?

The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.

Do you really regard braking in front of a sudden incline/decline as ridiculous?

These grids usually lie flat on the ground and in some areas are commonplace. Seems to me these sensors are still needing work. Perhaps they see horizontal lines and panic?

Cars running off the road for no good reason is ridiculous. It is indefensible to assert this is OK. Denying it or otherwise covering it up is definiteky ridiculous.

From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.

At least in the US we try to grandfather in old tech because the poors have enough problems without the government telling them they can no longer use their junk.

If a box was checked, a standard was met or a capacity was stated when the vehicle shipped the box remains checked more or less in perpetuity. This makes for some great internet arguments when the pearl clutching crowd tries to appeal to the authority of manufacturer statements with regard to older vehicles.

Indeed recently while going through the California smog check and registration process, the dealership told me I had open recalls which might block my registration. They didn’t, but clearly something like that mechanism exists.

This doesn’t unfairly harm people who already own vehicles.

The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.

If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences.

(I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)

In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.

When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).

You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.

Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall.

Would you react? Maybe swerve coz you’re startled?

Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.

(While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )

I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?

This is a particular challenge in California because our school zone signs specify that the limit only applies "when children are present."

Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.

I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.

Next trick: QR codes with cryptographic signatures, and a huge traffic jam when the traffic authorities forget to renew their certificate...

> So, it is still easier to

... hold them responsible in court for damage/injury/deaths caused by the actions their vehicles take which are not under control of the driver.

If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person.

A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes.

(I can dream, right?)

I can see a wide spectrum between my (admittedly over punitive towards corporations) ideas and this (somewhat intentional misinterpreted) idea of yours. But I do honestly think Uber and Tesla have evaded morally deserved punishments by blaming customers or contract staff who were pretty much doing what everybody would expect to be able to do using the technology as described.

"it's inevitable that even a responsible _person_ will screw something up" - but that doesn't mean we give them a zero consequence break if they kill or injure someone. (We do, however, distinguish between things like murder, manslaughter, and negligence-causing-death. And penalise differently, based at least partly on the ability of the punishment to discourage other people from doing similar things. Perhaps Uber was only guilty of negligence-causing-death for using minimum wage contractors as safety drivers. Perhaps Tesla was guilty of manslaughter by advertising as "autopilot" a thing that drives full speed into offramp dividers?)

I’d kinda like to see Tesla’s reaction if they were told that any “autopilot” crashes could result in the entire fleet of that model car being taken off the road, with Tesla responsible to all the owners for the disruption while a thorough investigation takes place, possibly a year or more.

It will be interesting to see how regulators treat autonomous vehicles. I have a feeling they will defer to the existing recall mechanism rather than completely grounding a fleet

If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.

For reference bugs in Boeing's plane resulted in the whole fleet being grounded.

In the large, it's always a shame to fix a good test case.