Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Nvidia DGX GH200: The First 100 Terabyte GPU Memory System. Parent comment
But all of that (trainable parameters, activations, optimizer state) is like 12 bytes per trainable parameter, not 80.
There are no comments on this comment yet.
But all of that (trainable parameters, activations, optimizer state) is like 12 bytes per trainable parameter, not 80.