Re:RedPajama v2 Open Dataset with 30T Tokens for Training LLMs
by yorwba on Tuesday October 31, 2023 @03:15AM
(#38081205)
The TinyLlama project is trying to do that pushing by training a small 1.1 billion-parameter model on 3 trillion tokens: https://github.com/jzhang38/TinyLlama
[ Parent
| on HN ]
There are no comments on this comment yet.