There is a philosophical debate underlying this. Take the analogy of a ML algorithm:

We know many algos are DESIGNED to be a black box, to be unexplainable. Red team iterates an incredibly effective algorithm that produces the desired outputs (with unseen risk built up as well). Blue team, in order to manage risk, is tasked with .. explaining the unexplainable process? An impossible task.

Is it possible that human/organizational processes can be also unexplainable?