Re:Σ-GPTs: A New Approach to Autoregressive Models
by hexaga on Friday June 07, 2024 @07:14PM
(#40613872)
> Temperature is a parameter for how you sample those logits in a non-greedy fashion.
I think temperature is better understood as a pre-softmax pass over logits. You'd divide logits by the temp, and then their softmax becomes more/less peaky.
Re:Σ-GPTs: A New Approach to Autoregressive Models
by qeternity on Friday June 07, 2024 @07:20PM
(#40613906)
Sure, my comment about softmax was simply about the probability distribution. But temperature is still part of sampling. If you’re greedy decoding, temperature doesn’t matter.
I think temperature is better understood as a pre-softmax pass over logits. You'd divide logits by the temp, and then their softmax becomes more/less peaky.
Sampling is a whole different thing.