Humans can make risk/reward calculations that computers can't. Top take an extreme example, a given set of snow and ice conditions might be bad enough that any reasonable person would not want to drive in them. In such a situation, any well designed self-driving car would refuse to drive. BUT my wife is pregnant, it's a difficult pregnancy with a high risk of death for her and the infant, and she's suddenly gone into labour weeks early in a freak snow storm.

A contrived example, yes, but the point is there may well be conditions in which, given a wider context, a human would do things a computer can't because it doesn't have the wider context. The thing is these contexts come up all the time in real life. A human can take responsibility, but a computer can't and a human that doesn't in detail know exactly how the computer is designed and will deal with different situations can't make a reasonable judgement about how far to trust the computer in such situations anyway.

Oh computers can make the risk/reward calculations, their criteria for risk and reward is wholly in the hands of who develops the logic.

and there in lies the problems for companies developing these systems. with current law they would be liable for any negative outcome. logs would be required to be kept and every decision would surely be pulled apart by whatever paid experts each side can find.

so without legislation, legislation the consumer must be very wary about, comes about limiting liability to such systems they will have to side with caution and even then every event is a lawsuit waiting to happen.

finally, would you or anyone you know trust a system which states it can do something you find adversely risky to actually do with you or your loved ones at risk? what would that take? it might simply be a decision of local values

I don't know; in many fields you're certainly right, but most of the autonomous vehicle field is software companies and I can't recall any instance of them facing a major liability.

So would we call it a loss of autonomy if a human occupant would only grant an override saying "Standing still is not an option!"? I guess you would want such a feature when there is no steering wheel, e.g. if there is a forest fire close behind you.

While a computer can't judge that situation and doesn't "care" if it is destroyed it very well can keep driving no matter what.

A forest fire is a really good challenge scenario. The threat isn't really anything to do with driving or road hazards as such, it's an out of context threat.

It's these out of context situations that are my long term issue with losing the steering wheel. A typical example would be arriving at a country fair and parking in a field. I want to park on the side of the field that's currently in sunlight, but over the next hour will be shaded by those trees, near the gate but not too near the toilets, but not next to that patch of wet ground.

What kind of UI does the car need to have for me to specify exactly how and where I want it to park, without me actually driving it? Frankly I think this is level 6+. Level 5 is usually described as a self driving taxi, but family vehicles are used for things no taxi would ever have to deal with.

As for parking in a field, I'd want it to drop me off closest to the entry and then head to wherever the site wanted it to park.

They work amazing when properly developed.

Growing up in a land with 4-5 months of winter, I was very comfortable in the snow and ice. Still scary as hell at times but the human body is remarkably adaptable to the environment. 30 feet visibility with snow and ice on the roads engages all the senses (except smell). Listening to the sound the tires are making helps know what they’re encountering. Inner ears helping know if a slide is starting. Sight constantly doubting itself and peering into the barely distinguishable road. All the nervous system helping with steering and the pedals. The road has a feel to it in such conditions that any man made system has quite a ways to go before engaging in it.

I do think it can get there. But it may need something better than our current AI tech, as well as some more sensors than lidar.

I also live in a US state where snow is the norm for that amount of time. I would argue "comfortable" is not safe and also, ultimately, not Level 5. There are some situations that are just not passable while maintaining safety. That's the reality and you are making risk choices that are subjective which cannot be understood by the AV. But we shouldn't assume because there will always be situations where a car will refuse to drive because of conditions that it's all for nothing.

I agree that I think it'll get there. But, like air travel there are limits and limits ground flights. When we drive a car we don't have a rigid pre-drive checklist, people in ATC watching out for weather and other aircraft, and then a company behind the scenes making the call on risk of life, money and brand reputation. Sure, weather is probably the hardest to deal with in knowing ahead of time if the trip can be completed with 100% success rate. I think the bigger problem is that as cars start to drive themselves, for people with little drive time throwing those people in the mix to take over will result in a very bad outcome. I have over 20 years of driving in ice and snow and still run across situations that are challenging or catch me off guard. But... What about that 17 year old who's car just told them to take over and they've never had to deal with driving in a storm ever? As we stop the task we lose the skill. Ultimately I feel that's the bigger issue holding things back until the manufacturers can get to Level 5.

> I agree that I think it'll get there. But, like air travel there are limits and limits ground flights.

+1 - And I feel like this is really what Waymo was saying.

> as well as some more sensors than lidar.

Like... cameras behind some kind of glass with special heaters and moving wipe thingies to keep the glass clean? I am sure I have seen something like this working somewhere.

Have some faith in your sensors. The point is they enable us to drive better than anything man made.

It's always a sad spectacle watching AI fans disparage humans when one looks at the miserable results of the various self-driving efforts. They should at least wait until they have something working decently.

Humans are so smart that they can't be modeled by AI. Humans are so dumb that they can't even design a system that does the same things that they do :)

I find it amusing that AI vs Human is actually 'A group of humans working on a technology' vs Humans. If we say AI sucks it actual that we probably aren't good at making certain type of software. Its funny when we talk about AI like it has some Sentience -- when all the onus on improving it and why it sucks is because of humans. Guess someone should start a 'AI shaming' hashtag! :D