News for hackers. Stuff that matters.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System by mirekrusin on Wednesday May 31, 2023 @03:23AM (#36135486)
You don't need to and can't really load all training data.

For LLMs you need to load single row of context size, that's vector of ie. 8k numbers, which is 32kB for single precision floats.

There are no comments on this comment yet.