Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by joakleaf on Tuesday June 06, 2023 @07:52AM (#36211274)

• The GPU in the base M2 has 10 GPU cores and 3.6 tflops

• The GPU in the M2 Ultra has 76 GPU cores and corresponds to 2x M2 Max, which has 13.6 tflops; so 27.2 tflops [1,2]

• RTX 4090 has 82.58 tflops [3] (overclocked can reach 100 tflops [4])

While more powerful NVidia cards are not "multiple orders of magnitude more". Rather it seems a 4090 is around 3-4 faster than an M2 Ultra GPU.

Keep in mind that the Apple Silicon chips also have low-precission Neural Engine circuits for inference of neural nets. For the M2 Ultra they claim 31.6 tops [1].

[1] https://www.apple.com/newsroom/2023/06/apple-introduces-m2-u...

[2] https://www.notebookcheck.net/Apple-unveils-M2-Pro-and-M2-Ma...

[3] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090.c3889

[4] https://videocardz.com/newz/overclocked-nvidia-rtx-4090-gpu-...

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by mosshammer on Tuesday June 06, 2023 @11:44AM (#36214607)

The RTX 4090 has 330.3 TOPS with INT8 precision, so for inference workloads it is still a magnitude faster

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by lostmsu on Tuesday June 06, 2023 @11:55AM (#36214765)

Since we are talking about AI, you should use Tensor Core Tops instead of regular flops. That number for 4090 is 330. So more than 10x of M2 Ultra

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by fyzix on Tuesday June 06, 2023 @02:21PM (#36216995)

Max to ultra doesn't scale linearly. Max tech on YouTube covered this extensively

[ Parent | on HN ]

Re:Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM by joakleaf on Tuesday June 06, 2023 @03:00PM (#36217498)

Probably true that performance doesn't scale linearly (and I do remember the Max Tech analysis of M1), but I don't have the tflops number for the M2 ultra.

For the M1 the stated values are:

  M1 Ultra [1]: 21 TFLOPs
  M1 Max [2]: 10.6 TFLOPS

Actual performance obviously depends on the type of work-load etc. E.g. Geekbench Ultra is only 50% faster [3]:

  M1 Ultra Geekbench 6: 150260 
  M1 Max Geekbench 6: 108198

[1] https://videocardz.com/newz/despite-apples-claims-m1-ultra-g... [2] https://wccftech.com/m1-max-teraflops-performance-higher-tha... [3] https://browser.geekbench.com/metal-benchmarks

[ Parent | on HN ]