News for hackers. Stuff that matters.
Re:RedPajama v2 Open Dataset with 30T Tokens for Training LLMs by yorwba on Tuesday October 31, 2023 @03:15AM (#38081205)
The TinyLlama project is trying to do that pushing by training a small 1.1 billion-parameter model on 3 trillion tokens: https://github.com/jzhang38/TinyLlama

There are no comments on this comment yet.