Re:The GPU is not always faster
by _zoltan_ on Wednesday December 11, 2024 @02:26PM
(#42391714)
get a GH100 on lambda and behold you have 900GB/s between CPU memory and GPU, and forget PCIe.
[ Parent
| on HN ]
Where are you seeing 900 GB/s?
The 900 GB/s are a figure often cited for Hopper based SXM boards, it's the aggregate of 18 NVLink connections. So it's more of a many-to-many GPU-to-GPU bandwidth figure.
https://www.datacenterknowledge.com/data-center-hardware/nvi...
this is bidirectional bw and it's gh200, not gh100
Re:The GPU is not always faster
by imtringued on Thursday December 12, 2024 @04:39AM
(#42397713)
That doesn't add up, because you're now exceeding the memory bandwidth of the memory controller. I.e. it would be faster to do everything, including the CPU only algorithms, in far away VRAM.