I can understand why this wouldn't happen from a business perspective, and it's also presumably not as simple to implement as I'm implying, but I can't think of a better way to get around the uncertainty of whether the car's operating in an expected way or not.
The "confidence meter" would almost certainly be 100% right up until it crashes into an obvious obstacle
Yeah, that's basically what I'm asking for, maybe a warning light at a certain threshold would be a better default, personally I'd still find a number more trustworthy though.
> The "confidence meter" would almost certainly be 100% right up until it crashes into an obvious obstacle
If image recognition algorithms have associated confidence levels, I'd be surprised something more complicated like road navigation was 100% certain all the time.
Unless you design the network to specifically answer "how similar is this?" and pair it with a training set and the output of another executing neural net, a "confidence meter" isn't possible.
This is one of the trickier bits to wrap one's mind around with neural networks. The systems we train for specialized tasks have no concept of 'confidence' or being wrong. There is no meta-awareness that judges how well a task is being performed in real-time outside of the training environment.
Humans don't suffer from this issue (as much) due to the mind bogglingly large and complex neural nets we have to deal with the world. Every time you set yourself to practicing a task, you are putting a previously trained neural network through further training and evolution. You can recognize when you are doing it because the process of doing so takes conscious effort. You are not 'just doing <the task>', you are 'doing <the task>, and comparing your performance against an ideation or metric of how you WANT <the task> to be performed'. This process is basically your pre-frontal cortex and occipital lobe for instance, tuning your hippocampus, sensory and motor cortex to perfect the perfect tennis swing.
When we train visual neural networks, we're talking levels of intelligence supplied by the occipital lobe , hippocampus, alone. Imagine every time you hear about a neural net that a human was lobotomized until they could perform ONLY that task with any reliability.
Kinda changes the comfort level with letting one of these take the wheel doesn't it?
Neural nets are REALLY neat. Don't get me wrong. I love them. Unfortunately, the real world is also INCREDIBLY hard to safely operate in. Many little things that humans 'just do' are the results of our own brains 'hacking' together older neural nets with a higher level one.
Calibrating the probabilities of machine learning algorithms is an old problem. By nature of disciminative algorithms and increasing model capacities, yes training typically pushes outputs to one extreme. There is a ton of information still maintained which can be properly calibrated for downstream ingestion, which anyone actually trying to integrate these into actual applications should be doing.
Most recently:
https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1802.03916
If for instance it failed to detect an obstacle how could it tell you that it failed detecting an obstacle? Or that it got the road markings wrong?
Given that the consequences for making a mistake might be people dying I certainly hope that's not how it works under the hood. Anything less than "I'm absolutely sure that I'm not currently speeding towards an obstacle" should automatically trigger a safety system that would cause the autopilot to disengage and give the control back to the driver, maybe engaging the emergency lights to let the other drivers know that something is amiss.
I wouldn't be surprised if in the moments preceding these crashes the autopilot algorithm was completely sure it was doing the right thing.
Of course these various crashes show that in fact the algorithm makes mistakes sometimes but I certainly hope that it's not aware that it's making mistakes, it just messes up while it thinks everything is going as planned.
Personally having tried Tesla AP1 iit was painfully clear I can never fully trust any assisted driving aid, they can fail at any point and I need to be ready to take over immediately. Particularly having to wrestle the steering wheel from AP1 as it slightly resists is reason enough to stop using it.
I think I'd find it a simpler metric to follow than the more qualitative information about car behaviour, as well as more useful to decide whether the car or I should be in control.
You're quite right about timing, I have no idea whether this metric would plummet quickly or gracefully drop so the driver would have time to take charge once a threshold was hit.
I want to stress that this is an underbaked idea and really just a thought experiment about what would make me personally feel more comfortable with driving a semi-autonomous vehicle.
This should be a law.
#1 job of any self driving system is to avoid obstacles. If you can’t do that reliably, it’s a fail!
It's just not good enough. Semi-autonomous cars can't be hitting stationary objects without reacting at all. If they fail at this basic task, then they shouldn't be allowed on the road.
You probably have 100ms to begin your action, 500ms to complete it.
Good luck with that if you're not perfectly alert.
The Queensland government estimates it takes drivers 1.5 seconds to react to an emergency [1] (I wouldn't be surprised if that was a 99th percentile figure, of course)
The only thing we can blame the victim for here is foolishly trusting Tesla's apparently crap software.
[1] https://www.qld.gov.au/transport/safety/road-safety/driving-...
If you have to be constantly at this level of alertness, what is even the point of having an autopilot?
Your cognitive load is lowered by Autopilot enormously. You can pay better attention to what’s going on around you and anticipate issues (say, a car that drivers weirdly in front of you) because you’re not busy with the low-level driving.
Part of that is that you see a difficult road stretch coming up and disengage the assist IN ADVANCE - becuase you’re not a complete moron, know the lane assist is for driving in highway lanes and the thing you’re approaching is decisively not one.
If you over-automate something you end up having to spend more time supervising and fixing it than if you'd just done the thing yourself in the first place.
Auto-pilot will make driving a lot more like flying is for pilots: 99% extremely dull, 1% terrifying.
A funny feeling like that, I’m ready to disengage immediately and then 1s is plenty of time. But I wouldn’t use Autopilot near an obstacle like that to begin with - anticipate, take over for the weird stretches.
Regardless, you're saying that if you're not paying attention every single second of the trip, instantly ready to rect to even the slightest deviation from your intended trajectory, then you're doing it wrong.
What, exactly, is the point of an auto-pilot system if you need to be in that state of absolute alertness?
Also unless certified on certain roads with clear markers, drivers should be required to have hands on wheels at all times.
I have been thinking that too. At what point do you decide that the autopilot is making a mistake and take over? That's an almost impossible task to perform within the available time.