Give GPT-4 about 10 or a dozen words from a meaningful sentence scrambled. Now tell it to unscramble those words to make a meaningful sentence. As long as you don't try too many tokens, the rate of meaningful completions is nearly 100%. The chances of that happening via random chance is to put it lightly, extremely small.
The only way for humans to solve this kind of problem is to think about the words before the output or in this case generation.
If you're giving it words from a sentence it is literally the baseline functionality of the text generator to choose the most likely next token, IE to recreate the most likely sentence from the words given.
This couldbe the least meaningfully revealing task I could possibly think of trying if I were trying to find some inner monologue, if gpt had to "think" on anything this would never be it.
Chance simply doesn't enter into the equation to any notable extent.
Rearrange (if necessary) the following words to form a sensible sentence. Don’t modify the words, or use other words.
The words are: access capabilities doesn’t done exploring general GPT-4 have have in interesting its it’s of public really researchers see since terms the to to what
A successful completion would be.
Since the general public doesn't have access to GPT-4, it's really interesting to see what researchers have done in terms of exploring its capabilities
The number of permutations of the 24 words in the pre-scrambled sentence without taking into consideration duplicate words is 24 * 23 * 22 * ... * 3 * 2 * 1 = ~ 6.2e+23 = ~ 620,000,000,000,000,000,000,000. Taking into account duplicate words involves dividing that number by (2 * 2) = 4. It's possible that there are other permutations of those 24 words that are sensible sentences.
For a language model to consistently make sensible predictions, it quite simply has to be able to "look ahead".
When the probabilities for the candidate tokens for the first generated token were calculated, it seems likely that GPT-4 had calculated an internal representation of the entire sensible sentence, and elevated the probability of the first token based on that internal representation.
I don't know what else to call looking backward, looking forward and then producing output anything other than a state.
If you want to see what this kind of completion would look like without much or any regard for a sensible completion of the sentence then just ask 3.5.
Also on my second try, I gave it a slightly invalid sentence. The prompt was..
Unscramble the following: waffles eat with like cheese you.
You like to eat waffles with cheese.
By the nature of number ranges there are infinitely many more wrong answers available to calculators performing addition and yet a simple ripple carry adder still promptly produces the correct output without lookahead, why this is compelling evidence?
Further, if it relied on internal state or lookahead why would it occasionally hallucinate appropriate tokens it wasn't provided?
If you want to believe looking ahead is unecessary, be my guest lol.
Humans would occasionally "hallucinate" words in too. For whatever reason(and this is verified independently of any of this), LLMs also appear to "get distracted"
"As I said before, the insights are not particularly original, I’m just raising awareness of issues."
If you are familiar in the subject matter, then that may have been your cue to stop reading.
(Don’t worry, I’m not equating the two… but this pathway does work.)
I'm with you on the point you'd like to make, but LLMs obviously do have state that we cannot see. In fact, there are two different levels of obvious state: one allows the LLM to (for example) respond to input instead of ignoring it, which we may characterize as "unlearned" behavior. The level people are more interested in is the "finished model", the set of associations it learns in training. It's state and we can't see it.
The internal state of the model, at those levels, doesn't change when you interact with it. But the state is there.
(There is also a "working state", the state the model must keep in order to output different tokens in sequence, and in order to at some point decide that it's finished responding to a prompt. This does change during your interaction with the model, but the change is not persistent.)
I guess that is some rock!
What made you think your analogy was bad?
I was not trying to say chatgpt is actually like a rock. But you knew that already - you just chose to ignore it in order to get the epic dunk.
In the same way if you understand the implementation of these types of language models you know that they simply cannot have inner monologues in the same way we do...the very concept of an inner monologue implies that there is some sort of state that we the user cannot see. Whether LLMs "think" or "want" or what thinking or wanting means etc can all be debated but whether the models have state that we cannot see is known to us: no. They do not.