News for hackers. Stuff that matters.
Re:33TB of text data for a 1T-parameter model by marginalia_nu on Sunday April 16, 2023 @02:17AM (#35587721)
While I agree with the conclusion, I don't think your argument actually supports it. Humans for the most part don't acquire language through books until they're mostly already fluent in their native language.

These LLMs are also trained on arbitrary books. When acquiring new languages, we use educational materials that are specifically created to facilitate an understanding of language. Not just arbitrary books in random order.

There are no comments on this comment yet.