This is basically the biggest research question the field of ML has. How to express useful prior knowledge and how to embed it in a network ?

The current answer mostly is "network architectures can express priors". That's what matt's convolutional net example is about.

This is clearly insufficient to solve the many still unsolvable-by-deep-learning problems, so the search is still on for more mechanisms.