This is essentially linear algebra, not behavioural psychology. I believe that one shouldn't draw such broad conclusions from a 1% improvement on some evaluation dataset.

I'm glad we're not tolerating attempts at pretending deep learning has anything to do with brains.

I was talking about the ability to use thin layers over top of broad learning. If neural networks are a realistic analogy at all, I think it fits. The roots of all this ML stuff is not straight from linear algebra even though much of the math is.

You're playing it extremely fast and loose with concepts like "low-level prerequisite knowledge" and how exactly does something "rel[y] on that knowledge", though. These aren't physical quantities like temperature where we -- as a species -- have the massive amount of low-level prerequisite knowledge that allows us to make rapid high level judgments that rely on that knowledge. The previous sentence is an example of how easy this reasoning is to abuse.

Could be possible. For example, when a child is deprived of any human and humane contact, they end up with a very very impoverished linguistic system as well as delayed development of all cognitive abilities. Could be because the lack the unsupervised training phase.