Megatron-Turing NLG 530B, the World’s Largest Generative Language Model

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by postalrat on Monday October 11, 2021 @04:35PM (#28832745)

CPU in kilohertz then megahertz then gigahertz then it stopped.

RAM in kilobytes then megabytes then gigabytes then it stopped.

[ Parent | on HN ]

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by sanxiyn on Monday October 11, 2021 @11:26PM (#28835738)

Yes for CPU, no for RAM. You can buy a computer with terabytes of RAM just fine. It's just expensive.

[ Parent | on HN ]

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by soganess on Tuesday October 12, 2021 @01:34PM (#28842549)

Expensive is a really relative term right about now... 64gb DDR4 LRDIMMs could be had for about 250$ each on eBay before the chip shortage. While that price is a "good-ish deal", it really wasn't unheard of. A search I just did returns more than a few hits...

Now compare to single GPU prices...

[ Parent | on HN ]

Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by manquer on Monday October 11, 2021 @04:48PM (#28832874)

Those are material science and physical limitations.

Number of parameters in a neural network is not really limited that way, doing useful compute with it is a different matter

[ Parent | on HN ]