News for hackers. Stuff that matters.

A comment on How Meta trains large language models at scale. Parent comment

Re:How Meta trains large language models at scale by refulgentis on Thursday June 13, 2024 @01:20AM (#40666197)
This breaks my brain, because I know Google trains it models on TPUs and they're seen as faster, and if they're better at inference, and can train, then why is Nvidia in a unique position? My understanding was always it's as simple as it required esoteric tooling
[ Parent | on HN ]
Re:How Meta trains large language models at scale by smueller1234 on Thursday June 13, 2024 @01:25AM (#40666236)
Multiple types of TPUs.

(I work for Google, but the above is public information.)

[ Parent | on HN ]
Re:How Meta trains large language models at scale by vineyardmike on Thursday June 13, 2024 @11:24AM (#40670808)
Because people generally don’t use TPUs outside of Google. The tooling is different, the access is metered through GCP, etc.

Nvidia is in a vaguely unique position in that their products have great tooling support and few companies sell silicon at their scale.

[ Parent | on HN ]
Re:How Meta trains large language models at scale by refulgentis on Thursday June 13, 2024 @01:07PM (#40672092)
Correct, I'm pointing out politely that's in conflict with the person I'm replying to.
[ Parent | on HN ]
Re:How Meta trains large language models at scale by koe123 on Thursday June 13, 2024 @12:51PM (#40671916)
Possibly naive but I very much view CUDA and its integration into ML frameworks being nvidias moat
[ Parent | on HN ]