News for hackers. Stuff that matters.
Re:The GPU is not always faster by _zoltan_ on Wednesday December 11, 2024 @02:27PM (#42391723)
GH100 can do 900GB/s HtoD.
Re:The GPU is not always faster by alecco on Wednesday December 11, 2024 @02:39PM (#42391891)
And both 3090 and 4090 can do 32 GB/s host-device. Not far from CPU-RAM. You only load the matrix once. The bandwidth for the matmul is orders of magnitude larger and happens all in device and mostly in cache.
Re:The GPU is not always faster by jing on Wednesday December 11, 2024 @08:08PM (#42395050)
No it can’t. That’s d to d
Re:The GPU is not always faster by saagarjha on Thursday December 12, 2024 @02:54AM (#42397176)
Well, device to device is technically doubled because you have a read and a write. But yes
Re:The GPU is not always faster by shaklee3 on Friday December 13, 2024 @12:38AM (#42406191)
No. D2D is arroubd 3TB/s
Re:The GPU is not always faster by _zoltan_ on Friday December 13, 2024 @07:24AM (#42407901)
no.

[root@gh200 nvbandwidth]# nvidia-smi | grep GH200

| 0 NVIDIA GH200 480GB On | 00000009:01:00.0 Off | 0 |

[root@gh200 nvbandwidth]# ./nvbandwidth | grep -E 'host_to_device_memcpy_sm|device_to_host_memcpy_sm' | grep ^SUM

SUM host_to_device_memcpy_sm 357.45

SUM device_to_host_memcpy_sm 352.05

[root@gh200 nvbandwidth]#

d2d is much higher:

[root@gh200 cuda-samples]# ./bin/sbsa/linux/release/bandwidthTest --dtod | grep -B1 32000

   Transfer Size (Bytes) Bandwidth(GB/s)
   32000000   2294.8
[root@gh200 cuda-samples]#