Biological models are going to have specific in built heuristics to learn specific things like language more easily.

Seems to me we should be training DL networks to adjust models to resemble models trained via backprop, but without access to backprop, and see what kinds of heuristics it provides.