News for hackers. Stuff that matters.

A comment on GPT-3 has no idea what it’s talking about. Parent comment

Re:GPT-3 has no idea what it’s talking about by skatesor on Saturday August 22, 2020 @11:32AM (#24244772)
You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else.

You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independently in order to produce something for others to copy.

[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by nmca on Saturday August 22, 2020 @12:05PM (#24245009)
This isn't true, sampling can produce novel output (and often does).
[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by Der_Einzige on Saturday August 22, 2020 @02:48PM (#24246263)
Uhhh, no. If your model is using subword tokenization like fasttext or... BERT with wordpiece (and the GPT-X models do this...), than you can generate entirely new words. Wasn't there a demo about doing exactly this a few days ago?

https://www.thisworddoesnotexist.com/

[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by skatesor on Saturday August 22, 2020 @05:02PM (#24247237)
Again, it's all coming from somewhere. Humans can react to stimuli in the natural environment and produce sounds and turn them into words. GPT has to be spoonfed a dataset a human made at some point.
[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by sjg007 on Saturday August 22, 2020 @12:29PM (#24245204)
Words it has to have seen,, but sentences no. It is generating new sentences. I think it can “name” things as well.
[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by nmca on Saturday August 22, 2020 @04:07PM (#24246879)
This is not true, BPE can back off to characters so arbitrarily character strings can be generated.
[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by c8g on Saturday August 22, 2020 @12:38PM (#24245264)
you have a reply from machine :)

https://philosopherai.com/philosopher/you-have-to-remember-t...

[ Parent | on HN ]