Re:Σ-GPTs: A New Approach to Autoregressive Models
by hexaga on Friday June 07, 2024 @07:14PM
(#40613872)
> Temperature is a parameter for how you sample those logits in a non-greedy fashion.
I think temperature is better understood as a pre-softmax pass over logits. You'd divide logits by the temp, and then their softmax becomes more/less peaky.
probs = (logits / temp).softmax()
It isn't. There is no guarantee that successive tokens will be comprehensible.
> Usually it outputs “logits”, which become a probability distribution when combined with a “temperature” parameter.
The logits are the probability distribution (well technically, you would apply softmax). Temperature is a parameter for how you sample those logits in a non-greedy fashion.