Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by senko on Tuesday June 06, 2023 @07:09AM (#36210927)

> seriously, measuring graphics cards in gigabytes is like measuring battery life in volts. it's the wrong unit.

Yes, but, if you need 48GB to run inference on a model, and you only have 24GB available, you don't get to enjoy the tflops difference.

If nVidia released somewhat lower-performance GPUs but with more VRAM, we could talk. But they're not stupid :)

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by nightski on Tuesday June 06, 2023 @10:00AM (#36212898)

You can get an A6000 for cheaper than an M2 Ultra. That said it's still only 48GB.

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by senko on Tuesday June 06, 2023 @11:33AM (#36214414)

I'm not sure if it's cheaper when you compare the whole machine price, not just GPU, since you can't buy piecewise with the Mac.

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by nightski on Tuesday June 06, 2023 @12:27PM (#36215237)

The GPU is $4500. That gives you $2000 to build a machine (which you can build a top of the line consumer PC for cheaper than that, I just did recently w/128GB DDR5 and Zen 4 7950X3D).

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by senko on Tuesday June 06, 2023 @01:55PM (#36216612)

Cool if you can find it at that price. On my side of the pond, it's around €6,000.

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by sterlind on Thursday June 08, 2023 @01:37AM (#36237665)

yes you do. you can use DeepSpeed. it's only like 2x slower.

[ Parent | on HN ]