It seems to me that you could treat a sufficiently accurate model (be it a deep net or otherwise) as the solution to an unknown, governing physical law, and attempt to identify the law from the model, post-hoc.

I'm not sure if there's been work done in the domain of pattern recognition, but here's an overview where in an example, Navier-Stokes is identified from data:

https://sinews.siam.org/Details-Page/data-driven-discovery-o...