> For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.

There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.