I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe).
> Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence.
> Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.
"Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent.
But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.
Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly.
We don't use supervised learning on LLMs. There is no way we can train an LLM by using human supervisors to rate every output. It works for humans since humans learn with so few examples, supervising a kids language learning doesn't take much effort, a single person can easily manage it while doing it for an LLM would consume the whole worlds workforce for many years.
Untrue. Many cultures don't much speak to their children and they turn out just fine. It's fairly evident Language learning is primarily unsupervised.
https://www.scientificamerican.com/article/parents-in-a-remo...
I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well.
Even with parents that aren't laissez-faire, they're correcting a handful of times a day tops which can't even begin to account for full language proficiency.
Children need correction yes but in a supervised way? There's no indication of that.
How do you know? Humans learn from extremely few corrections, often just a single time is enough for the human to correct themselves and learn it for life.
Kids learn the bulk from hearing and seeing examples. Then they fine tune that with the help of their parents and peers correcting them when they do it wrong. You can learn language and skills without those corrections, but it will be full of errors and mistakes that are easily corrected with an outsider pointing it out for you.
If corrections weren't vital for human learning then humans wouldn't be so eager to correct each other when they make mistakes. There is no other purpose for it than to help them learn and get better. So your belief that those corrections doesn't do anything seems very unfounded, I'd like to see extremely strong evidence to the contrary to believe you here.
I've been around little children being raised. I have some grasp on how much supervised correction is happening. It's very very little.
You don't seem to get it. Children know thousands of words fluently by age 5. This is consistent across many cultures and circumstances. You would need over a handful of corrections per day right from the day of birth to square how much children know.
And this is all assuming one correction per word which as much as humans are great learners seems like a very dubious assumption.
I don't know what else to tell you other than this obviously isn't happening.
I assume much less than one correction per word. Corrections are to fix when the kids made a mistake when they mimic others, kids doesn't make mistakes for every word so you don't need to correct every word.
As I said, bulk of learning is from mimicking, but they will make a lot of mistakes when mimicking so to become good they need corrections. When I see parents with kids I see them correcting their kids all the time, correcting misunderstandings or fixing pronunciations or grammar issues etc. You can notice that parents use a specific voice when they correct the pronunciation, and the kid gets that the parent tried to correct their pronunciation from it, that happens quite a lot with small kids. You don't say "this is how to pronounce X", our genes seem to encode a way to communicate pronunciation to others to help correct mistakes.
Kids also correct each other in this way, which helps make the learning more robust even when adults aren't around. All humans has instincts to correct others so I don't think any culture fails to do this.
This isn't an uncommon claim, but it's certainly far from settled science. Complex language is an impressive capability of human cognition. But I think the claim that cognition itself is a byproduct of language proves too much. Obviously the products of cognition that are easiest for us to understand are linguistic, because language is the primary standardized means by which we communicate. To me, this is just more legibility bias, though. There are plenty of cognitive processes that people find impossible to explain in words, but still nonetheless exist
I always thought that language definitely shapes our understanding of the world, but at much more fundamental level, I believe language falls apart to teach us anything. For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with), but we just know them through our interactions with others and environments. If you try to teach a kid a concept of temperature "hot", you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it and when mom comes along and say something like "do not touch it it is very hot". There are things we just cannot understand through language.
You shouldn't. When you learn a foreign language dictionaries can be a great help and it doesn't matter if they use circular definitions. Well it can matter in a situation like Stanisław Lem described with his "sepulkas"[1], but even then the definitions were a red warning. Ijon Tichy just failed to understand it.
> you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it
I learned English by reading books mostly. In most cases I had no easy access to a dictionary and inferred meanings of words by a context. In some cases I failed to infer meaning but nevertheless in each case I managed to get some idea about a word. I remember some surprises when I found a real meaning of a word and it was not exactly what I thought. Or even some ideas that felt new and inspiring for me before I connected them to ideas I learned long before that in my native language. I'm like an English LLM myself, because I never used English in a real world context, only to read texts and to write comments in English. To this very moment I cannot talk about some topics in Russian, because I do not know Russian words to talk about them.
All this experience led me to doubt an idea that you can understand language only by connecting it to reality. I believe you can. Your language will be disconnected from reality and it can be a disaster probably if you try to apply it to reality, but you can talk a lot, participate in philosophical debates and it doesn't matter your understanding is different. It is like redness of red: do you see red as I do and does it matter?
> There are things we just cannot understand through language.
We cannot link our senses with language without burns. But it doesn't mean we cannot understand. For example, you can watch other people touching really hot boiling kettles. If you never touched anything hot you will not understand their pain, but you'll know they are in a pain and you'll know of a special nature of that pain. You can learn all the important intricacies of interaction with hot objects by just watching. Or by reading. Your understanding will be limited still, but in a lot of cases it doesn't matter.
Though of course we coming very near to a debate what is understanding. Is it a human-centric definition, that boils down to "only a human can understand", or it is more relaxed and rely on a pragmatic idea, like does your understanding enables you to make right choices. If LLM doesn't have a body that can feel pain, it doesn't matter if LLM cannot know how pain feels.
Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs.
Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs.
Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence.
Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.