News for hackers. Stuff that matters.
Re:RedPajama v2 Open Dataset with 30T Tokens for Training LLMs by mauriceweber on Tuesday October 31, 2023 @03:51PM (#38090495)
yes, small clarification: the 1TB per dump refers to the head+middle partition of the dataset and includes the text documents and the quality signals. There is another ~700GB for the minhash signatures and 1-1.5TB for the documents in the tail split.

There are no comments on this comment yet.