News for hackers. Stuff that matters.

A comment on Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs. Parent comment

Re:Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs by jbarrow on Tuesday January 07, 2025 @01:55PM (#42625906)
Primarily concerned about the memory bandwidth for training.

Though I think I've been able to max out my M2 when using the MacBook's integrated memory with MLX, so maybe that won't be an issue.

[ Parent | on HN ]
Re:Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs by ryao on Tuesday January 07, 2025 @02:46PM (#42626537)
Training is compute bound, not memory bandwidth bound. That is how Cerebras is able to do training with external DRAM that only has 150GB/sec memory bandwidth.
[ Parent | on HN ]
Re:Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs by jdietrich on Tuesday January 07, 2025 @04:38PM (#42627882)
The architectures really aren't comparable. The Cerebras WSE has fairly low DRAM bandwidth, but it has a huge amount of on-die SRAM.

https://www.hc34.hotchips.org/assets/program/conference/day2...

[ Parent | on HN ]
Re:Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs by ryao on Tuesday January 07, 2025 @06:05PM (#42628800)
They are training models that need terabytes of RAM with only 150GB/sec of memory bandwidth. That is compute bound. If you think it is memory bandwidth bound, please explain the algorithms and how they are memory bandwidth bound.
[ Parent | on HN ]