News for hackers. Stuff that matters.
Re:Why GPT-3 Matters by visarga on Tuesday July 21, 2020 @02:26AM (#23904751)
500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCrawl as training data.

It's like the proverbial witch's pot where they put everything in and out comes the magic.

Re:Why GPT-3 Matters by YeGoblynQueenne on Tuesday July 21, 2020 @06:25AM (#23905744)
Yes, that's an apt metaphor. Magic, indeed.