Re:Does GPT-2 Know Your Phone Number?
by minimaxir on Sunday December 27, 2020 @11:57AM
(#25552325)
I ran an experiment through GPT-3 with just that frame: https://github.com/minimaxir/gpt-3-experiments/tree/master/e...
[ Parent
| on HN ]
Re:Does GPT-2 Know Your Phone Number?
by andreareina on Monday December 28, 2020 @01:29AM
(#25557213)
What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.
Those are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token.
Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
Re:Does GPT-2 Know Your Phone Number?
by andreareina on Monday December 28, 2020 @12:51PM
(#25561307)
Thanks, didn't twig onto the fact that you linked a subtree of the whole repo. Weird that even with the nonzero temp the AskReddit prompt went a bit loopy.
> https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
Oh my goodness that is absurd in the most delightful way. Thanks for sharing that.