News for hackers. Stuff that matters.
Re:Does GPT-2 Know Your Phone Number? by hyperpape on Sunday December 27, 2020 @10:57AM (#25551832)
Of course it isn't how the human mind works.

You can ask a person "did you come up with that sentence, or is it something you read?" And they can answer that question. The answer isn't perfectly reliable, but it isn't completely unreliable either. You can't do that with GPT-2/GPT-3.

Big picture: the human mind is a machine of sorts. It's not the same kind of machine as any form of machine learning we've developed so far.

Re:Does GPT-2 Know Your Phone Number? by ben_w on Sunday December 27, 2020 @11:09AM (#25551947)
> The answer isn't perfectly reliable, but it isn't completely unreliable either.

How would we be able to tell how reliable it is in general?

"""The left brain of people whose hemispheres have been disconnected has been observed to invent explanations for body movement initiated by the opposing (right) hemisphere, perhaps based on the assumption that their actions are consciously willed.""" - https://en.wikipedia.org/wiki/Neuroscience_of_free_will#Rela...

Re:Does GPT-2 Know Your Phone Number? by alisonkisk on Sunday December 27, 2020 @08:49PM (#25555979)
The point is that humans have a basic ability to make specific coherent observations about themselves and the world. GPT can't do that except occasionally by accident in the middle of a muddle.
Re:Does GPT-2 Know Your Phone Number? by _flux on Sunday December 27, 2020 @04:40PM (#25554496)
In this case you could do it with GPT-x as well, if you have access to the learning material (so the OpenAI could do it when providing the model as a service): grep the learning data after-the-fact.

No idea if it would be feasible to integrate this information directly into the model. With my limited understanding of neural networks, this seems difficult.