Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by mk_stjames on Tuesday June 06, 2023 @11:27AM (#36214330)

I recently sat and thought about it, and I made the bold claim to a friend that the development of CUDA over the last 15 years.. just that activity, as a whole, is one of the largest use of human intelligence to ever be focused on any one thing in history. It's less people than worked on getting to the Moon, but if you measure the project in PhD-person-years it is probably off the charts. And if you include all the engineers and scientists that the program affects in the same way you would have included all the subcontractors for the Apollo program, it is way, way bigger.

I think the only reason CUDA isn't talked about like the monumentally important human milestone in technological development that it is, is that it is a pretty abstract thing that is difficult for laypeople to visualize.

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by kkielhofner on Tuesday June 06, 2023 @12:23PM (#36215188)

This is an excellent point.

I've been accused of being an Nvidia "fanboy" when I touch on this. I attempt to explain:

"Nvidia made the very hard, very expensive commitment to developing and supporting CUDA 15 years ago when this space was in it's infancy (non-existent). They SUNK incredible resources into this gamble/vision to universally support CUDA on every chip and every platform - for 15 years. They didn't achieve their position through shady dealings or luck, they earned it with a decade and a half of investment, focus, and execution."

Granted, they do somewhat abuse the position they have now (as often noted on HN and elsewhere). At the risk of whataboutism I ask: "Show me a corporation that wouldn't. Do you think if AMD had their market share they'd be nice, cuddly good guys?" Microsoft, Intel, Nvidia, Standard Oil, the Phoebus cartel, etc - it goes on and on. Always has and always will. Of course I'm not saying it's a good thing, it's just a fact of the real world.

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by mrtranscendence on Tuesday June 06, 2023 @03:02PM (#36217531)

So what if other companies would similarly abuse their position? If it doesn't excuse Nvidia's behavior in some way then it hardly seems relevant to point out. We still need more competition in this space, or Nvidia parts will become increasingly bad deals (as we've seen for consumer GPUs).

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by kkielhofner on Wednesday June 07, 2023 @12:31PM (#36229186)

I'm not saying it's a good thing, I'm simply pointing out it's more-or-less universal standard behavior for people and corporations to abuse positions of power unless checked externally - anti-trust action, checks and balances within government, other regulation, etc. As noted the recurring tendency on HN and elsewhere to paint Nvidia as some kind of unique diabolical actor is very strange.

How are Nvidia consumer GPUs a bad deal? When comparing top of the line cards (I'm not going to bother looking elsewhere in the product lines) for 60% more cost (yes, significant) with an RTX 4090 you get:

- Performance that walks all over the 7900 XTX[0].

- The ability to use it to self-host, experiment, learn, etc a never-ending range of ML applications that (as discussed) "just work" as a docker pull away.

With an RTX 4090 you could have stunning gaming performance one minute and then seconds later be running a local LLM, etc. That is tremendously more value than the 60% price difference. If all I wanted to do was gaming and I was price sensitive I'd save myself the $600 and be happy with an AMD GPU. But, looking at market share[1] (at least 80% across desktop gaming and GPGPU) either the value of AMD GPUs is a little known secret (it isn't) or consumers, with the ability for choice, overwhelmingly see the value in Nvidia GPUs.

[0] - https://techguided.com/7900-xtx-vs-rtx-4090/

[1] - https://wccftech.com/q3-2022-discrete-gpu-market-share-repor...

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by touisteur on Tuesday June 06, 2023 @12:35PM (#36215401)

The sheer volume of work of 'just' creating and maintaining a C++-like compiler, highly performant cuBLAS, cufft, cuSOLVER, cuDNN with very high perf, all by their lonesome in closed source shows how far one would have to go to even eat parts of their lunch.

We (who want a real alternative to NVIDIA) either find a way to pool global resources applicable to this level kind of effort on all new accelerator architectures, or we wait for them to stumble, Intel-like. Intel and AMD not pooling their resources on this is self-defeating.

[ Parent | on HN ]