Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by rfoo on Wednesday May 31, 2023 @05:08AM
(#36136158)
It's not per parameter, you also need to hold activations for back prop to work.
[ Parent
| on HN ]
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by mirekrusin on Wednesday May 31, 2023 @08:33AM
(#36137586)
You need activations for inference as well.
But all of that (trainable parameters, activations, optimizer state) is like 12 bytes per trainable parameter, not 80.