What Is Temperature in NLP?

Re:What Is Temperature in NLP? by mlazos on Monday March 13, 2023 @08:04AM (#35134523)

I get that temperature comes from the connection to physics but this seems like an overly complex explanation to a relatively simple formula. If you have a softmax and you multiply the input by some factor, if that factor forces the exponent to approach zero you’re approaching the uniform distribution, and if you make the exponent approach infinity you get the arg max. Curious if others feel the same way.

[ Parent | on HN ]

Re:What Is Temperature in NLP? by rickdeckard on Monday March 13, 2023 @08:22AM (#35134725)

Are you suggesting that your explanation is more simple? (genuine question)

I find "temperature" quite intuitive as a metaphor in that context. It doesn't even require me to think of the formula to understand "the higher the temperature, the more it will create a stew of the results"

[ Parent | on HN ]

Re:What Is Temperature in NLP? by albystein on Monday March 13, 2023 @12:28PM (#35138011)

This is exactly how I understand it. Although, I feel the thermodynamics analogy might be more intuitive to most people

[ Parent | on HN ]