Re:LLMs Are This Close to Destroying the Internet
by coffeebeqn on Sunday April 14, 2024 @12:37PM
(#40032354)
Is it really true though? If they scrape HN today I don’t think their sampling would be that “polluted” compared to 2022.
If you were to teach a LLM to play chess today - wouldn’t you use fully AI generated data because that’s by far the best data ?
Re:LLMs Are This Close to Destroying the Internet
by johntb86 on Sunday April 14, 2024 @03:40PM
(#40033769)
Isn't all of recorded human history up until 2022 enough to train a pretty smart ai? It'll miss out on future trends, but some targeted training on trusted news sources should be enough. Eventually languages will change enough that the old training data will seem dated, but that's going to take a while to happen.
That seems like it might make the training process untenable.