News for hackers. Stuff that matters.
Re:Modern language models refute Chomsky’s approach to language by MAXPOOL on Tuesday June 13, 2023 @03:01PM (#36315156)
I agree. Question has not been settled.

20 year old human has

* heard ~220 million words, talked 50 million words.

* read ~10 million words.

* experienced 420 million seconds of wakeful interaction with the environment (can be used to estimate the limit to conscious decisions, or number of distinct 'epochs' we experience)

From a machine learning perspective human life is surprisingly small set of inputs and actions, just a blip of existence.

Re:Modern language models refute Chomsky’s approach to language by pphysch on Tuesday June 13, 2023 @03:05PM (#36315235)
If you reduce it to bytes, yeah, humans are unreasonably efficient learners. But that's not how it works, there's loads more information embedded in a real human speech context.
Re:Modern language models refute Chomsky’s approach to language by IshKebab on Tuesday June 13, 2023 @03:50PM (#36315940)
LLMs learn from unlabeled data. Children definitely do not. There's a huge difference. I would not be surprised if LLMs could learn a lot more efficiently if they had carefully constructed training data, with video and sound.

But also humans have been speaking for so long it's silly to imagine we don't have some evolved language structures in the brain. I don't know why anyone would single that out for skepticism while not questioning e.g. the brain structures for sight, sound, emotions, navigation, etc.

Re:Modern language models refute Chomsky’s approach to language by MAXPOOL on Tuesday June 13, 2023 @05:21PM (#36317564)
> some evolved language structures in the brain.

That's Chomsky's argument. A small set of constraints for organizing language.

Re:Modern language models refute Chomsky’s approach to language by 8note on Tuesday June 13, 2023 @05:32PM (#36317763)
This is testable, right?

Put a kid from one language tradition in a spot with a different language tradition, and they won't be able to learn it.

Eg. kids with native mandarin speaking parents adopted to native Indo-European parents fail at learning English, and will be better at learning Mandarin than their peers with Indo-European heritage.

Re:Modern language models refute Chomsky’s approach to language by mcguire on Tuesday June 13, 2023 @07:14PM (#36318964)
That doesn't work. Children will learn whatever language they are exposed to. That is why Chomsky and others talk in terms of a universal grammar.
Re:Modern language models refute Chomsky’s approach to language by IshKebab on Wednesday June 14, 2023 @04:46AM (#36323182)
I don't think anyone is suggesting that the brain has structures that are specific to a particular language.