Re:Does GPT-2 Know Your Phone Number?
by golergka on Sunday December 27, 2020 @10:20AM
(#25551593)
I only took a beginner course in ML over 5 years ago, so this probably is a stupid question, but does this mean, the trained GPT-2 model encodes the source text into it's parameters somehow? Is this resilient — will it still remember it if we randomize the weights just a little tiny bit? Will it remember it if we clear a small portion of parameters?
Does the human memory work the same way?