Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by moffkalast on Wednesday May 31, 2023 @08:30AM
(#36137557)
"The best time to start training a 100T param model was 30k years ago, the second best time is now."
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by rowanG077 on Wednesday May 31, 2023 @08:53AM
(#36137780)
That's obviously false under the assumption computing power will increase as it has in the past.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by moffkalast on Wednesday May 31, 2023 @09:07AM
(#36137925)
For the uninitiated, it's a tree planting quote.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by EvgeniyZh on Wednesday May 31, 2023 @11:38AM
(#36139584)
Probably the best time to start train 100T model is never
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by samstave on Wednesday May 31, 2023 @12:04PM
(#36139936)
What if you could train an AI with a desired outcome to their answers?
I.E. ; "answer this question where the outcome is the most beneficial to quality of life"
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by moffkalast on Wednesday May 31, 2023 @12:26PM
(#36140223)
> life
A strange game. The only winning move is not to play. unplugs self
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by chromoblob on Wednesday May 31, 2023 @12:27PM
(#36140243)
A properly designed AI agent would do exactly that.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by EvgeniyZh on Wednesday May 31, 2023 @01:20PM
(#36140936)
I'll take your question further: what if we have unlimited data (say some crazy rich RL environment or way to produce high quality and diverse synthetic data)? You still have to get these 1e30 FLOPs. Lets say you can connect 100 of these bad boys together with 40% utilization, with total 4e19 FLOPs/s. Assume also Moore's law keeps working indefinitely. When should we start training 100T model on it to get is as early as possible? We wait x years and the start training on machine with 4e19*2^(x/2) FLOPs/s. Turns out the answers is ~16 years, after which we'll have 1e22 FLOPs/s and 1e30 FLOPs will take another 3 years.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by samstave on Wednesday May 31, 2023 @12:00PM
(#36139885)
I am not going to be embarrassed for the following Q ;
Please ELI5 where I can have a glossary of AI/ML terms - where do I get fluency in speaking about Tokens, Models, Training, Parameters, etc...
Please dont be Snarky - This is info that everyone younger than I am needs as well.
Is there a Canon? Where is it?
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by EvgeniyZh on Wednesday May 31, 2023 @01:09PM
(#36140758)
I have no idea tbh. I learned these a while ago (~7 years ago), and the materials I used then are heavily outdated and also I won't be able to remember what they were. I guess any intro course to deep learning should talk about these. Stanford ones used to be good. Maybe someone else can be more useful about it.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by rocmcd on Wednesday May 31, 2023 @01:38PM
(#36141145)
I'd recommend starting here:
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
It's pretty lengthy but doesn't require a PhD to understand. If you can get to the end of it you'll have a much better understanding of what's going on.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by eamsen on Wednesday May 31, 2023 @03:26PM
(#36142892)
At the risk of sounding snarky, https://chat.openai.com would be a good introduction, followed by books, which GPT could recommend.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by Crye on Thursday June 01, 2023 @01:23AM
(#36147837)
To be honest, I'd start with some introduction to Transformer YouTube videos. They'll cover a lot of these terms and you'll then have a better understanding to find additional resources.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System
by causalmodels on Thursday June 01, 2023 @02:59AM
(#36148298)
> Rule of thumb is that you need ~20 tokens per parameter.
That rule of thumb is wrong. The chinchilla paper has it anywhere between 1 and 100 tokens per parameter.
But in this case what is really ridiculous is the compute requirement. The required compute for optimal model growth roughly quadratically (both your model and your data grow linearly). So for 100T model you need 1e30 FLOPs. This machine gives you 1e18 FLOPs per second. It will take 30k years to train this model on one of these (or 30k of these to train it in a year, but then utilization will start kicking in).