Re:Does GPT-2 Know Your Phone Number?
by andreareina on Monday December 28, 2020 @01:29AM
(#25557213)
What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.
[ Parent
| on HN ]
Those are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token.
Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
Re:Does GPT-2 Know Your Phone Number?
by andreareina on Monday December 28, 2020 @12:51PM
(#25561307)
Thanks, didn't twig onto the fact that you linked a subtree of the whole repo. Weird that even with the nonzero temp the AskReddit prompt went a bit loopy.
> https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
Oh my goodness that is absurd in the most delightful way. Thanks for sharing that.