News for hackers. Stuff that matters.

A comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language Model. Parent comment

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by TeMPOraL on Tuesday October 12, 2021 @07:27AM (#28838433)
Wonder how this architecture is limiting the space, though - all biological brains train continuously. Our DNNs are more like a brain upload snapshot that's always run for one cycle and then rebooted.
[ Parent | on HN ]
Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by abecedarius on Tuesday October 12, 2021 @02:52PM (#28843566)
Yeah, that's worth exploring more. It's just that it's not safe to depend on "real neurons are complicated, and therefore artificial nets of simple units won't have transformative capabilities".
[ Parent | on HN ]