Actually that's pretty much exactly the theory I've seen suggested a few times, Harold Klawans is a big believer for a start. He makes a good case in his books that human intelligence is very much the result of the fact that much of our brain growth occurs post-birth as our bodies increase in size, which makes us quite different from most other species. It's not just that our brains absorb knowledge as we grow into adults, but transform themselves radically in order to gain particular skills (particularly language processing).