Megatron-Turing NLG 530B, the World’s Largest Generative Language Model

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by abecedarius on Monday October 11, 2021 @09:57PM (#28835222)

Emulating a neuron != taking a comparable part in a computation. Probably the former is a lot more complex. For instance, an artificial net can take advantage of backpropagation in a separated training phase -- that's a lot of complexity that's factored out of the runtime phase.

[ Parent | on HN ]

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by TeMPOraL on Tuesday October 12, 2021 @07:27AM (#28838433)

Wonder how this architecture is limiting the space, though - all biological brains train continuously. Our DNNs are more like a brain upload snapshot that's always run for one cycle and then rebooted.

[ Parent | on HN ]

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by abecedarius on Tuesday October 12, 2021 @02:52PM (#28843566)

Yeah, that's worth exploring more. It's just that it's not safe to depend on "real neurons are complicated, and therefore artificial nets of simple units won't have transformative capabilities".

[ Parent | on HN ]