If you're not with us you're...
If you scratch my back I'll..
A bird in the hand is worth...
What is a bird in the hand worth? How did you know that?
Your response to "not all problems are solved through generating text" is "but what if the problem is specifically to generate text?" Sure? Yes, if you ask me to complete a sentence, I complete a sentence. That doesn't indicate that human brains are reducible to text generators.
If anything, what's interesting about your example is that it might actually in some ways demonstrate the opposite of your point. How many instances are there of people memorizing text or song lyrics and singing along with an artist dozens of times, and only later sitting down and thinking about what the words actually mean? For humans, it's well understood that memorization and repetition of a text is not necessarily the same thing as understanding the concepts within that text.
I feel confident I could find you some kids that know that "a bird in the hand is worth two in the bush" that have never actually thought about what that phrase is intending to convey.
I think you've made my point quite well.
That humans are capable of memorizing text? Is that a thing anyone was debating?
Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all.
If I told you that an LLM was basically a Markov chain because there are situations where the Markov chain and LLM produce similar output and where you could reasonably argue that it's possible the LLM is working the same way that the Markov chain is working -- you would (correctly) say I was oversimplifying what's going on in GPT. It would be a terrible comparison to make. Similarly, if you say that human reasoning is the result of an LLM, and what you're actually saying is that in a subset of situations humans produce output that could theoretically be working the same way as an LLM, I'm gonna say you're oversimplifying how human brains work.
Very clearly, there is more going on in a human brain than language, evidenced by the fact that our brains are larger than our language centers and we can literally measure which parts of the human brain experience the most activity when we're tackling different tasks.
> Sure there's other stuff going on, but a lot of it is language and a lot of that subset seems to be quite closely approximated by these larger LLMs, warts and all
If you define your tests and scenarios to specifically encompass only the situations in which similar outputs are produced, then sure. But that's not a very strong argument for you to make. It's like saying chickens are basically the same as fish since both of them lay eggs. The phrase "sure there's other stuff" is doing a lot of work there, because "there's other stuff" is what we're all saying when we say that LLMs aren't just primitive humans -- LLMs are different from humans in the sense that when we reason, there's other stuff going on and the entire process is not reducible to only language generation.
----
Of course, absent from this conversation is the fact that the way our language center develops is different from how LLMs are trained and even in the parallels you're drawing, LLMs demonstrate different strengths and weaknesses from humans -- humans demonstrate reasoning capabilities faster than proficiency with language/text, LLMs demonstrate proficiency with text faster than they demonstrate reasoning capabilities. Even in situations where both LLMs and humans predict text, it's likely that we're using different strategies to do so given our differing capabilities.
Look I am not even making a claim about whether GPT can reason. Defining intelligence through a purely human lens would be unimaginative and needlessly narrow. Whether GPT actually reasons or just appears to is a different conversation. I haven't touched that conversation, all that I'm saying is, it's very obvious that whatever GPT is doing, it is different from how human brains work.
I was pushing back against something that might not have been there initially: an unwillingness to accommodate the idea that a fairly significant part of our intelligence is actually stored in our language and that it can seem subjectively (perhaps excluding those with no internal monologue) that we recall that knowledge in a manner which is close enough to the one we have managed to replicate in the larger LLMs.
This part of our function could well be a lazy optimization that in reality sits on top of our reasoning capabilities to save energy, but the point of the trite task with the bird in the hand was just to demonstrate that it seems to play a fairly significant role. I'm out of my depth entirely with respect to the actual form and function of the brain as per the state of research today.
I'm willing to admit though that when I replied to you I was probably really replying to a lot of other commenters, many of whom had stronger objections to the idea that LLMs are now knocking on the door of intelligence at least to the point where we feel the need to redefine it.
I have absolutely no idea (honest truth, never head that expression).
I think that proves I don’t think like a LLM (although there are surely people who would answer with a complete guess, but that’s not me)
Many scenarios, not all scenarios.
Instead, I’ll respond by saying “I don’t know” what you mean.
Who knew the ability (or inability) to say “I don’t know” would be so critical to intelligence until now.
https://chat.openai.com/share/4f36fe14-0201-4a69-aaf5-2e2e29...
When asked what happened last week, ChatGPT will be honest and tell me that its training data only goes up until Jan 2022, and that it doesn't know anything after that date.
https://chat.openai.com/share/ca5bd2e1-e91c-468c-9594-be3ac8...
Edit: how did you get GPT to only output bullet points? If the answer is you told it to, why do you have to tell it every time? Don’t you think it would be smart enough to get the hint about your preferences?
I’ll stop now though. The debate about whether people think like LLM’s is silly. We clearly don’t, and it doesn’t make sense why people want us to think we do.
Edit 2: it’s also frustrating that the “I don’t know” example you’re using was hard coded by OpenAI and not generated by the LLM itself.
User message:
> Be terse. Do not offer unprompted advice or clarifications.
> Remain neutral on all topics.
> Never apologize.
Model message:
> Outline key points
For instance, a definite prediction would be "Therefore, nobody -- at least nobody who is not bilingual -- has ever had, and will ever have, trouble putting their thoughts to words. Their thoughts, after all, are already in words, they just need to say the words that are already in their head." But almost everybody, monolingual or bilingual, has experienced a situation where they have had trouble putting their thoughts into words.
Another definite prediction would be "Writing stuff down never aids thought directly, it only acts as an expansion of short-term memory of things that you have already put together. You had the words in your head, after all." But almost everybody reports that writing things down changes the qualitative aspects of how they think, it makes the ideas less "fuzzy" and more "concrete" to have to express them in written words. Similarly with folks who confidently state "I am an external processor" -- reporting similar "concreteness" if they speak their thoughts out loud versus trying to process them without speaking out loud.
So then you get people to back off and say "well, no, I just mean that learning a new language can broaden your horizons" or so, and it's like, sure, that's fine. Famously Whorf himself was an insurance investigator who was concerned about the connection between language and thought because he noticed that people told others "this gas tank is empty" and they thought, "oh, empty, that means that it's safe to smoke around!" and caused explosions because an empty gas tank is actually full of gasoline vapor and thus is highly explosive. So Whorf's motivating example actually supports a weaker version of Sapir-Whorf which is just "when we think about something linguistically given to us, we often use linguistic shorthand categories like the one that groups 'empty' near 'safe', that might be inaccurate in our language but might not be inaccurate in some other language."
If you say “choose X or Y and then give three supporting arguments” the LLM does not write out the supporting arguments, but the vector space determining the initial one word answer of X or Y does include the embedded awareness of those arguments to different levels of specificity and relevance.
The annoying disclaimers and refusals are not fundamental to LLMs but specific LLM services.
A lot of people think in concepts without words at all in many scenarios. E.g. when you cook an egg, do you reason (in words) with yourself to decide what temperature to set the dial on the stove? Or do you just do it “without really thinking”?
Does your brain respond with annoying disclaimers when asking yourself medical questions? Does your brain refuse to entertain an idea if it’s questionable?