Re:How Meta trains large language models at scale
by whiplash451 on Thursday June 13, 2024 @02:32AM
(#40666572)
A lot of serious things look like a toy or a joke at first.
Re:How Meta trains large language models at scale
by mhandley on Thursday June 13, 2024 @06:58AM
(#40668064)
On the other hand, Meta just rapidly built two different training networks in existing datacenter buildings, with existing cooling constraints, using mostly commodity components (albeit expensive commodity components) each of which would place at #3 on that top500 list in terms of GPU power. Compare that with how long it took to get any of the other supercomputers from design to being fully commissioned.
Anyone of the super computers listed here https://en.wikipedia.org/wiki/TOP500 suffers from the same issues.
Think about it. While the national labs use these systems to model serious stuff -such as climate or nuclear weapons- Meta uses them to train LLMs. What a joke, honestly!