News for hackers. Stuff that matters.

A comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language Model. Parent comment

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by rolisz on Monday October 11, 2021 @04:19PM (#28832576)
Mixture of Experts, aka not all 10 trillion parameters are used at the same time, just a subset that is an "expert" on the "task at hand".
[ Parent | on HN ]
Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by jobigoud on Tuesday October 12, 2021 @02:28AM (#28836697)
That would also describe an organic brain though.
[ Parent | on HN ]