News for hackers. Stuff that matters.
Re:LLMs Can't Do Probability by 6gvONxR4sf7o on Thursday May 02, 2024 @02:09AM (#40233192)
Nope, i’m describing prompt 1. The output of the model is a distribution over tokens, which is then sampled by the system to get the next word/token. This is what people mean when they talk about the logits in these models. So if you ask it for one single sample, you hope it might give you a logit vector that corresponds to 80% ‘left’ token and 20% ‘right’ token (assuming those words are single tokens, but it all still works otherwise, but is less concise to explain). When it then autocompletes, it samples from that distribution (a weighted coin flip, in your terms).

So it’s just neat that the weights in the coin flip don’t match what is asked for.

Re:LLMs Can't Do Probability by intended on Saturday May 04, 2024 @03:31AM (#40255698)
Just to check we are on the same page -

The output is the probability that x is the correct n+1 token based on the input of n tokens.

Your point reads as: the output will be a probability distribution where token n+1 has a chance to be 80% left and 20% right.

Reading your statement as given, this is not what the output of the LLM will be.

When it autocompletes the output will be ‘left’.

The LLM will not be doing a coin flip at this stage, since it’s prediction is based on text not random number generators.

The input vector constrains it to 80% left. Since it’s training data is human text, this essentially constrains the first output token to left 100% of the time.

If you try to have it provide tokens n+2,n+3… etc in the same output, then it will start spitting out right.

More generally - there isn’t any coin flip, just the next token being predicted based on the training corpora.

So when you say that it’s taking from a probability distribution, are you considering the use of capabilities beyond token prediction?