News for hackers. Stuff that matters.

A comment on GPT-Neo – Building a GPT-3-sized model, open source and free. Parent comment

Re:GPT-Neo – Building a GPT-3-sized model, open source and free by rich_sasha on Monday January 18, 2021 @07:18AM (#25821003)
I think this is exactly right, and indeed this is a lot of the value. "Content-generation" is already a thing, and yes it doesn't need to make much sense. Apparently people who read it don't mind.
[ Parent | on HN ]
Re:GPT-Neo – Building a GPT-3-sized model, open source and free by throwaway6e8f on Monday January 18, 2021 @07:33AM (#25821135)
People don’t read it, search engines do.
[ Parent | on HN ]
Re:GPT-Neo – Building a GPT-3-sized model, open source and free by masswerk on Monday January 18, 2021 @10:10AM (#25822483)
BTW, we should mandatorily tag generated content for search engines in order to exclude it from future training sets.
[ Parent | on HN ]
Re:GPT-Neo – Building a GPT-3-sized model, open source and free by hundchenkatze on Monday January 18, 2021 @03:43PM (#25826405)
Apart from that, hopefully the people building training sets use gltr or something similar to prevent training on generated text.

http://gltr.io/

[ Parent | on HN ]
Re:GPT-Neo – Building a GPT-3-sized model, open source and free by a1369209993 on Tuesday January 19, 2021 @01:48AM (#25830705)
Well, hopefully, someone will come up with a language model that picks words based on GLTR purpleness. Automated data collection deserves automated data.
[ Parent | on HN ]