I think it could be useful to distinguish systems which are required to automatically explain every decision they recommend as a matter of routine, versus systems which are not, but still are required to support post-mortem investigation of serious failures.