See my earlier answer - I think that it is all about the data. And most things humans design have a similar "correlation of parts" due to the human preference for symmetry. Not all the images are face on headshots - lots of running/action, off centered, etc. ImageNet is a very "real" dataset in that sense.

Entropy isn't the right term here - low entropy would imply something about the compressibility of each image based only on whether it was a machine or a natural object, which I don't think is case.

Well, if the effect is just experimental error, I agree that no explanation is needed.

I was using entropy in the information theoretic sense, as you might assign a measure of entropy in bits to each character in a language. If part of an animal is "less surprising" I'd say it contributed less to the entropy of the whole thing. Maybe that's too woolly.

There is a difference between complex mechanical and organic objects, though. See the universal machine tools illustrated on this page: http://www.lathes.co.uk/adcock%26shipleycombination/

There is no way you could identify the whole machine based on seeing a small part if an identical part recurs in many different machines. It's not holographic in the way animals are.