I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself.
(And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)
The fly-by-wire control system ordinarily prevents stalls, but it had disengaged due to an iced over sensor and was operating without stall protection. The plane stalled at 38,000 feet and fell into the ocean.
They should have had plenty of time to correct the stall, but one pilot was pulling back his control stick (the opposite of what they needed to do), and since the two sticks aren't physically linked the other pilot didn't know he was doing that.
It's one of the topics discussed in the podcast that I linked above.
We're still in pretty early days here, so I don't know if Watson's advice is at a point where that could be true. In the meantime, I do like the idea of having an unaided doctor and the computer program evaluate it independently, then afterward say "Ok, let's see what the computer thought" before coming to any final conclusions. Maybe it'd help avoid any dangerous recommendations. Or maybe people would say "Well the computer has a lot more data than me, it's probably right." I think that depends on whether the differences are errors similar to what a human would make, or if they're the "oh my god how did it even come up with this result" nonsensical error that are easily spotted by human review.
Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.