hynky
Re:
on Tuesday January 06, 2026 @05:31PM (#46519752)
HF recently released a blog about their pre-training dataset from PDFs.
Alongside that they also shared a graph how much PDFs they were able to fetch by date, and the old internet seems to be mostly dead.
Re:
on Tuesday January 06, 2026 @01:38PM (#46516547)
HuggingFace recently released a report on their FinePDFs dataset, with this saddening report on state of current availability of old internet.
Re:Smollm3: Smol, multilingual, long-context reasoner LLM
on Tuesday July 08, 2025 @03:35PM (#44503264)
yes, both core web datasets are publicly available as well as the rest