Re:LLMs Can't Do Probability
by intended on Saturday May 04, 2024 @03:31AM
(#40255698)
Just to check we are on the same page -
The output is the probability that x is the correct n+1 token based on the input of n tokens.
Your point reads as:
the output will be a probability distribution where token n+1 has a chance to be 80% left and 20% right.
Reading your statement as given, this is not what the output of the LLM will be.
When it autocompletes the output will be ‘left’.
The LLM will not be doing a coin flip at this stage, since it’s prediction is based on text not random number generators.
The input vector constrains it to 80% left. Since it’s training data is human text, this essentially constrains the first output token to left 100% of the time.
If you try to have it provide tokens n+2,n+3… etc in the same output, then it will start spitting out right.
More generally - there isn’t any coin flip, just the next token being predicted based on the training corpora.
So when you say that it’s taking from a probability distribution, are you considering the use of capabilities beyond token prediction?
The output is the probability that x is the correct n+1 token based on the input of n tokens.
Your point reads as: the output will be a probability distribution where token n+1 has a chance to be 80% left and 20% right.
Reading your statement as given, this is not what the output of the LLM will be.
When it autocompletes the output will be ‘left’.
The LLM will not be doing a coin flip at this stage, since it’s prediction is based on text not random number generators.
The input vector constrains it to 80% left. Since it’s training data is human text, this essentially constrains the first output token to left 100% of the time.
If you try to have it provide tokens n+2,n+3… etc in the same output, then it will start spitting out right.
More generally - there isn’t any coin flip, just the next token being predicted based on the training corpora.
So when you say that it’s taking from a probability distribution, are you considering the use of capabilities beyond token prediction?