Re:The GPU is not always faster
by _zoltan_ on Wednesday December 11, 2024 @02:27PM
(#42391723)
GH100 can do 900GB/s HtoD.
[ Parent
| on HN ]
And both 3090 and 4090 can do 32 GB/s host-device. Not far from CPU-RAM. You only load the matrix once. The bandwidth for the matmul is orders of magnitude larger and happens all in device and mostly in cache.
No it can’t. That’s d to d
Well, device to device is technically doubled because you have a read and a write. But yes
No. D2D is arroubd 3TB/s
no.
[root@gh200 nvbandwidth]# nvidia-smi | grep GH200
| 0 NVIDIA GH200 480GB On | 00000009:01:00.0 Off | 0 |
[root@gh200 nvbandwidth]# ./nvbandwidth | grep -E 'host_to_device_memcpy_sm|device_to_host_memcpy_sm' | grep ^SUM
SUM host_to_device_memcpy_sm 357.45
SUM device_to_host_memcpy_sm 352.05
[root@gh200 nvbandwidth]#
d2d is much higher:
[root@gh200 cuda-samples]# ./bin/sbsa/linux/release/bandwidthTest --dtod | grep -B1 32000
Transfer Size (Bytes) Bandwidth(GB/s)
32000000 2294.8
[root@gh200 cuda-samples]#