If a bank denies you credit, it has to explain why. Not "AI told us so".
If police arrests you, they have to explain why, not "AI told us so".
If your job fires you, it has to explain why, not "AI told us so".
etc.
My claim is that we do have adequate explanations of human behavior and cognition for many purposes. We use them in our everyday life. Without them we couldn't function. We use them in court. Without them we could never ascribe responsibility. And we do all of this without having perfect knowledge of constituent parts. And we find this all acceptable, for the most part, for practical judgement and decision making.
The idea that we must have an exhaustive account of causality in AI whatever to make reasonable assertions about the causality behind is behavior is not true. The reason it is not true is because we do not require this anywhere else in the agentic world.
"When AI is used, its decisions must be explicable in human terms. No 'AI said so, so it's true'". Somehow the whole discussion is about how human mind cannot be explained either.
Yea, the decisions made by the human mind can be explained in human terms for the vast majority of relevant cases.
It doesn't
> it means that even if AI makes better decisions (name your metric), we would deny the use if those decisions if we could not sufficiently explain them.
Ah yes, better decisions for racial profiling, social credit, housing permissions etc. Because we all know that AI will never be used for that.
Again:
If a bank denies you credit, it has to explain why. Not "AI told us so".
If police arrests you, they have to explain why, not "AI told us so".
If your job fires you, it has to explain why, not "AI told us so".
etc. etc.
EDIT: it might not even be useful to insist on explainability if the results are better. We did not and do not do that in other areas.
For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.