News for hackers. Stuff that matters.
Re:$1T in Tech Stocks Sold Off as Market Grows Skeptical of AI by marcyb5st on Saturday November 08, 2025 @11:03AM (#45857630)
There's a limit to that according to: https://www.nature.com/articles/s41586-024-07566-y . Basically, if you use an LLM to augment a training dataset it will become "dumber" every subsequent generation and I am not sure how you can generate synthetic data for a language model without using a language model
Re:$1T in Tech Stocks Sold Off as Market Grows Skeptical of AI by yorwba on Saturday November 08, 2025 @11:36AM (#45857950)
Synthetic data doesn't have to come from an LLM. And that paper only showed that if you train on a random sample from an LLM, the resulting second LLM is a worse model of the distribution that the first LLM was trained on. When people construct synthetic data with LLMs, they typically do not just sample at random, but carefully shape the generation process to match the target task better than the original training distribution.