Re:Why GPT-3 Matters
by visarga on Tuesday July 21, 2020 @02:26AM
(#23904751)
500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCrawl as training data.
It's like the proverbial witch's pot where they put everything in and out comes the magic.
It's like the proverbial witch's pot where they put everything in and out comes the magic.