Indeed. I wonder if the networks could learn a more and more general description of the concepts in the hierarchy when going up that hierarchy. E.g. there's a bunch of tiger species [0] each with a specific underlying model, but some traits are common to all tiger species. And some traits are common to, say, carnivores. Could you share parts of the model/network via that hierarchy, e.g. that a tiger inherits some parts of the model for carnivores etc.

For example, can the 'stripe model' or 'fang model' be shared among tigers and carnivores etc.

I tried something similar for my thesis, but this was before the advent of DNNs etc.

[0] http://conceptnet.io/c/en/tiger?rel=/r/IsA&limit=1000