News for hackers. Stuff that matters.
Re:Why GPT-3 Matters by BenoitEssiambre on Monday July 20, 2020 @11:31AM (#23898782)
>All without having been explicitly trained in those tasks.

Can you elaborate on what kind of training data was used here? I'm curious.

Re:Why GPT-3 Matters by visarga on Tuesday July 21, 2020 @02:26AM (#23904751)
500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCrawl as training data.

It's like the proverbial witch's pot where they put everything in and out comes the magic.

Re:Why GPT-3 Matters by YeGoblynQueenne on Tuesday July 21, 2020 @06:25AM (#23905744)
Yes, that's an apt metaphor. Magic, indeed.