News for hackers. Stuff that matters.
Re: by jrmg on Tuesday March 03, 2026 @02:00AM (#47229083)
I have the same worry about LLMs in general - I know that ‘model collapse’ seems to be an unfashionable idea, but when the internet’s just full of garbage (soon?…), what are we going to train these things on?
Re: by tehjoker on Tuesday March 03, 2026 @09:26AM (#47232838)
They moved away from raw text and are now working with verifiable synthetic data (eg math, games, code) to improve general reasoning.