News for hackers. Stuff that matters.
Re:The Ultra-Scale Playbook: Training LLMs on GPU Clusters by maxrmk on Wednesday February 19, 2025 @04:24PM (#43107902)
Ok this is way too long to read in one sitting, but it looks incredible? I've been looking for resources like this for """real""" model training at scale. The authors have worked on BLOOM (the first >100B parameter model), starcoder and fineweb so they actually know this space.

I wish it were not a single giant webpage, but I guess I can deal with that.

There are no comments on this comment yet.