But we consider humans intelligent.
Can you expand on that a little?
Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
Which is more probable?
a) Linda is a bank teller. b) Linda is a bank teller and is active in the feminist movement.
It is an example of lack of mathematical (really probability) know how, not naïve pattern matching.
Obviously, people rephrase the question into : is there more chance that she's a feminist or that she isn't ?
The fact is, you can infer things about people based on things you know about people. I can pick a random user on HN, and knowing they're a user of HN, I can say it's probable that they work in technology. We don't need to bring statistics into it and turn it into a math problem.
a) Hank has written us a human interest problem b) Hank has written us a human interest and a probability problem.
I don't think people are simply wrong about the Linda problem, I think they're imprecise about which question they're answering, and more or less think they're answering a question about what chances that Linda is a feminist vs what are the chances she's a bank teller not only using the givens+relevant priors about people but also their priors about what kind of question they're answering. It isn't "no real reasoning", it's just not high resolution enough to be technically correct by the standards of a constructed probability problem.
You can argue LLMs are also not quite high resolution enough and I'd accept that. In my mind the question is what it would take to get some kind of ML software to a place where if you trained it on enough probability problems it would be able to evaluate the Hank problem above, including the issue of whether (a) and (b) are actually independent. ;)
I, like most people intuitively answered b). Given the explanation on the Wikipedia page I went "oh, of course, yeah", but then I thought about why I'd answer b) given that I'm fairly familiar with basic probability.
If you give me two options and ask me to pick between them, my brain is usually going to assume it's not a trivially true problem.
Language needs context for any sense to be made of it.
As a result of the above, reading the question, the intuitive reading makes the answer choices
a) Linda is a bank teller (implicitly, a bank teller NOT active in the feminist movement)
b) Linda is a bank teller and is active in the feminist movement.
This question is one of language, context, and interpretation, not of people failing to understand basic probability.
I suspect that if you prime people to excise interpretation of the question by presenting it as the following, the majority of people would guess correctly:
---
"Consider the following two statements:
1) Linda is a Bank Teller
2) Linda is active in the feminist movement
Which is more likely?
a) 1
b) 1 ^ 2"
a) Linda is a bank teller (and NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement
What you want it to be asking here is:
a) Linda is a bank teller, and may or may not be active in the feminist movement b) Linda is a bank teller, and is active in the feminist movement
Most college students have taken quite a few multiple-choice tests (particularly in the US, high-schools train for standardized multiple-choice tests). The question isn't asking what the mathematicians seem to think it's asking, because it's format conveys extra restrictions.
That some tech people think that human intelligence can be reduced to such "textural mechanics" betrays a lack of depth of understanding and even appreciation of the deep and complex world within which we find ourselves. Our written corpus is but a particular reflection of this reality - the shadows on the wall in Plato's cave if you will.
The irony here is that us humans (both individuals and collectives) experience a very "Plato's-cave-esque" version of reality, pretty much by design. The same Ancient Greeks built a functioning civilization that gave the world logic and physics and philosophy while believing utter nonsense about themselves and the world. We, the enlightened modern humans, also believe all kinds of nonsense about ourselves and the world, and have careful defense mechanisms against exploring anything beyond the "shadows on the wall" that we're allowed to observe by our tribal authorities. How many people still watch the "news", or even read "horoscopes", for instance? We are designed by evolution to simulate a tiny biased shadow/hallucination of reality that helps us/our tribes to thrive, and that includes self-delusions and collective blindness towards obvious BS (so we could organize around imaginary, but evolutionarily beneficial causes).
If it were interacting with the real world in some way (i.e. had 'eyes' and 'hands'), with a learning reward system, then maybe deeper patterns could get encoded.
I have some unique synonyms I know which are not in dictionaries or commonly used but I know when I have used them in conversation they understand.
I ask the LLM to provide definitions for WordA and WordB. -They come back essentially the same.
I ask the LLM if the two definitions are synonymous. -The LLM responds with yes.
I then ask if WordA is synonymous with WordB. -It replies no.
I expect the above is solvable through pattern recognition with tokens. I think given enough information it becomes effectively indistinguishable and practically the same to us. The above is just a testament to how it is getting there... Since it delivers intellect it will be called intelligent, but it doesn't appear to me to be approaching it like human intelligence. Though maybe given enough information, or through different modalities, like the real world as the other commenter says, it will model some deeper coherency.
Dogs have behavior, which hints at them understanding some concepts (threat, food, etc).
LLVM neural nets can only express sentences, but that's not the interesting part.
-- Some silly movie
What a terrible analogy. It would be like finding the brain is run by entirely random sampling processes and lacks any memory.
Concluding that electric signals in the brain can't be intelligence because it's just electricity is a typical pitfall if one does not realize that electric signals in the brain are about as likely to be caused by intelligence as they are to be causing it. Those signals is just one model offered by physics nowadays, which says nothing about the causal direction between those and things like consciousness.
Do you have proof of this? Who is to say we aren't anthropomorphizing other humans? They're just manipulating electrical charges after all.
Phrased another way, if LLMs actually did manipulate concepts, how would their performance differ? What is a tangible example of the former and of the latter?
The fantastic part is its ability to work at the concept level. This technology proved to be efficient at building some graph of concepts and relations simply be feeding it huge amounts of token. For example with the moon hoax, it is able to synthesize in two sentences the main points of the hoax theory.
It still has limitations, but that's the major step everyone is impressed with.
Saying "but it's just a markov chain underneath, it's not intelligence" would be like discovering electric signals in the brain and concluding it can't be intelligence because it's just electricity.