DeepMind’s New Language Model, Chinchilla

Re:DeepMind’s New Language Model, Chinchilla by sillysaurusx on Monday April 11, 2022 @11:13AM (#30989942)

Welcome to the party! I joined ML because I realized I could help. You can too. I bet you’re both already thinking of clever ways to deal with massive models from an infrastructure standpoint. That’s just one of hundreds of interesting problems.

[ Parent | on HN ]

Re:DeepMind’s New Language Model, Chinchilla by native_samples on Monday April 11, 2022 @01:57PM (#30992418)

Is 100GB of parameters really that large? 128GB of RAM on a server class machine is not unusual. Seems such a model could fit entirely in RAM.

[ Parent | on HN ]

Re:DeepMind’s New Language Model, Chinchilla by kristjansson on Monday April 11, 2022 @06:27PM (#30995916)

To elaborate on the sibling comment: main memory is much bigger, but CPUs are much, much slower. It would be a challenge to merely run a model like this on CPU, and totally infeasible to train one. So the challenge is to fit into the memory of a single GPU you can afford, coordinate multiple GPUs, or efficiently page from main memory into GPU.

[ Parent | on HN ]

Re:DeepMind’s New Language Model, Chinchilla by andbberger on Monday April 11, 2022 @02:01PM (#30992475)

GPU memory is generally much smaller and more expensive

[ Parent | on HN ]