News for hackers. Stuff that matters.

A comment on How Meta trains large language models at scale. Parent comment

Re:How Meta trains large language models at scale by ChuckMcM on Thursday June 13, 2024 @01:02AM (#40666129)
The bits on the bus go round and round!

There is a lot of interesting yet unpublished work on 'data center' scale compute complexes. It was a rabbit hole I fell into several times while at Google.

[ Parent | on HN ]
Re:How Meta trains large language models at scale by falcor84 on Thursday June 13, 2024 @08:46AM (#40668987)
They do publish some of that, or at least they used to. In particular "The Datacenter as a Computer" [0] was a very interesting read.

[0] https://research.google/pubs/the-datacenter-as-a-computer-an...

[ Parent | on HN ]