News for hackers. Stuff that matters.

A comment on OpenAI’s CEO Says the Age of Giant AI Models Is Already Over. Parent comment

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by sottol on Monday April 17, 2023 @03:59PM (#35605762)
But what are you mostly "teaching" the LLM then? Mundane everyday stuff? I guess that would make them better at "being average human" but is that what we want? It already seems that prompting the LLM to be above-average ("pretend to be an expert") improves performance.
[ Parent | on HN ]
Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by dougmwne on Monday April 17, 2023 @05:43PM (#35607021)
This whole conversation about training set size is bizarre. No one ever asks what’s in the training set. Why would a trillion tokens of mundane gossip improve a LLMs ability to do anything valuable at all?

If a scrape of the general internet, scientific papers and books isn’t enough, a trillion trillion trillion text messages to mom aren’t going to change matters.

[ Parent | on HN ]