News for hackers. Stuff that matters.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System by gmueckl on Wednesday May 31, 2023 @02:34AM (#36135198)
Not the GP, but I believe that they are talking about the size of the training data set in relation to the model size.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System by mirekrusin on Wednesday May 31, 2023 @03:23AM (#36135486)
You don't need to and can't really load all training data.

For LLMs you need to load single row of context size, that's vector of ie. 8k numbers, which is 32kB for single precision floats.