Current set of tools in explainable AI do allow deep neural nets to tell why they gave the response that they did to a certain extent.

Contact me if you want to see it applied to text based DL models, time series based DL models or Image based DL models.