The models require orders of magnitude more data to learn language than children do, and children's task is harder as they have to segment speech themselves.

The comparison isn't so straightforward, since children's language-learning environment is multimodal and extremely rich. They're exposed not just to sequences of words, but also to behaviors, emotions, social interactions, and the physical world.

Yes, I believe arguments like GP vastly underestimate the amount of data involved in children's learning. All the data coming into their sensorium, from heard language to physical movement feedback, contributes to training all their skills, from physical movement to language.

I think you just explained the situation, even if you didn't realize it.

LLMs process millions of times more text data and use millions of times more compute power to learn a language, mostly because we know how to do that technologically.

Children use much less text data, but also use a lot of other kinds of data, including interacting with other humans, to learn language. We have no idea how that works. And, on top of that, people can interact with pets of various species in the same way as children, but those pets never learn language.

So, something strange is going on that children can do, but that LLMs and puppies can't. Probably, part of it is that human children are much better at learning by copying what adults are doing, and part of it is that humans evolved to learn and use language. The details of what's going on there is what linguistics is about.

You're forgetting the millions of years of evolution that "pre-trained" child brains for this task.

I agree that language is innate.

Yes but it can be both innate and built on statistical models, contrary to Chomksy.

The idea that there is some level of statistical inference involved in language acquisition is not contrary to generative thought, look at Charles Yang's work or Lidz or a fair number of other people. Chomsky just thinks this is uninteresting so he is dismissive of it.

The central question remains whether there are categorical rules built into the brain for grammar (plus "some" statistical inference possibly), or whether the entirety of our grammatical processing is statistical. It is entirely possible that evolution encoded statistical priors into our brains with grammar overall being a fully statistical process. Therefore, the fact that children learn more quickly isn't really hard evidence one way or another.

[delayed]

Well, Chomsky wouldn’t be happy, that’s the point of OP; he explicitly argues against this.

But the statistics only work well at vastly unrealistic scales of data, and the learning pattern doesn't match children's behaviour at all.

A statistical model is approximating the brain at some level.

LLMs (at the autoregressive pretraining stage) are trained to predict what an (adult) human who could have said preceding words would have said next. It requires learning more than language.

A newborn brain is not a blank slate, it comes with a very rich prior (as Chomsky himself would argue), for example the ability to rapidly acquire language, control a body, etc.

An LLM must construct all of its grammatical structure, world models, etc. from scratch during pretraining.

LLM pre-training is more like recapitulating evolution of the mammalian brain than childhood development.

Of course children are not a blank state. My point is to argue that LLMs are not an argument against UG.

I agree that is something we need to learn a lot more about, but a human child's brain does not appear to be necessary to learn language and I argue the evidence supports that.

The question isn't "are neural networks just like human brains?" It is "can statistical models generate and understand language?"

The answer to the actual question under discussion appears to be a resounding yes. It does not appear that learning language is innate in the way that Chomsky has argued.

It may be that learning language as efficiently as human children will involve some innate property that we have not yet discovered. But that is a perfect example of moving the goal posts.

The hardest theory to disprove is one an entire field livelihood rests upon