https://google-research.github.io/seanet/audiolm/examples/
Linguistics is useful fiction. It doesn't accurately describe the complexity of language anymore than Newton's laws accurately describe gravity. You'd think people would understand that now with symbolic approaches failing this sort of stuff after decades of trying but i guess not. We don't know how we work. We really really don't.
The obsession with how we think we work over methods that are actually working is really something to behold.
Amen.
Humans don't start at zero.
But that's only true if you assume humans start at zero and the millions of years of natural selection that gave us our incredible brains gives no advantage to learning, which seems facially absurd. The toddler probably sees less data in that five years than the LLM sees in a few months, but it's worth remembering how rich human sensory data is, smell is heavily tied to memory and is incredibly complex, taste and touch sensitivity in the mouth is high enough that toddlers stuff new things in their mouth to understand them better, human hands are incredibly sensor dense, and that's before we get to the classics of 1000fps vision consisting of partially high definition and partially upscaled low definition video, or audio that contains huge amounts of information not only about material properties, energy magnitudes, or space layouts and shapes, but also a huge amount of semantic data through speech and language for most children. But unlike the LLM, they didn't start with a random selection of numbers, they started with a human brain which is incredibly impressive. Frankly, it is genuinely surprising that LLMs are able to rival us as well as they do considering how much less complex they are than our brains.
>it’s not that the pronouncements of linguists and philosophers are useless in the face of it, they’re just in a different domain.
I go back and forth on whether this is true. I see the argument, but it also makes intuitive sense that study of an AI system whose intelligence comes entirely from human language can probably yield insights about human language. It doesn't seem any less plausible than studying the way slime molds feed on oat flakes to gain insights into public transport design and infrastructure, which is something that we have done to significant success.
don't agree. the amount of written words we feed in, yes. but a tottler is a multimodal system including not only audio/images but actually video, plus other senses like touch or smell. This complex input 24/7 is a lot of data that happens, and some brain relations to classify the world in form before it can spell out a label (word) for it.
Even just looking at a simple word (like "apple"), the human already has a lot of contextual information about it, mostly how its presented and how the presenter frames it (like yummy, tasty fruit! like other foods in the fridge, ...), and then people say the word physically pointing at the thing.
My guess thats an order of magnitude more total input than our LLMs get solely via text or other standalone channels for training.
Which one do you eat? Which one does an LLM eat?
First of all, a baby does not passively listen. It sees and touches and smells and feels, and moves and makes sounds, and observes reactions, and correlates all that with the sounds it hears.
Secondly, with the above in mind, why on Earth would you expect it to "do absolutely nothing of any use"? It feels obvious it'll start honing in on language, with even better grounding (but worse performance) than a LLM. And I fully expect there to be a to merge such model with an LLM, the same way people combine LLMs with image generators.
Don't feed it a corpus of text! Ensure it's data is the opposite of a large language model, i.e. zero language.
Let it passively listen to the same sounds and speech as a baby hears in its first five years, over five years of operation.
Observe it do absolutely nothing of any use.
Get back to me about any useful comparisons with linguistics or philosophy of language.