News for hackers. Stuff that matters.

A comment on GPT-3 has no idea what it’s talking about. Parent comment

Re:GPT-3 has no idea what it’s talking about by Der_Einzige on Saturday August 22, 2020 @02:48PM (#24246263)
Uhhh, no. If your model is using subword tokenization like fasttext or... BERT with wordpiece (and the GPT-X models do this...), than you can generate entirely new words. Wasn't there a demo about doing exactly this a few days ago?

https://www.thisworddoesnotexist.com/

[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by skatesor on Saturday August 22, 2020 @05:02PM (#24247237)
Again, it's all coming from somewhere. Humans can react to stimuli in the natural environment and produce sounds and turn them into words. GPT has to be spoonfed a dataset a human made at some point.
[ Parent | on HN ]