Re:How to Scale Your Model: How to Think About GPUs
by charleshn on Wednesday August 20, 2025 @10:31AM
(#44962320)
Yes, 450GB/s is the per GPU bandwidth in the nvlink domain.
3.2Tbps is the per-host bandwidth in the scale out IB/Ethernet domain.
Re:How to Scale Your Model: How to Think About GPUs
by jacobaustin123 on Wednesday August 20, 2025 @12:09PM
(#44963195)
I believe this is correct. For an H100, the 4 NVLink switches each have 64 ports supporting 25GB/s each, and each GPU uses a total of 18 ports. This gives us 450GB/s bandwidth within the node. But once you start trying to leave the node, you're limited by the per-node InfiniBand cabling, which only gives you 400GB/s out of the entire node (50GB / GPU).
Re:How to Scale Your Model: How to Think About GPUs
by xtacy on Wednesday August 20, 2025 @01:06PM
(#44963748)
Is it GBps (gigabytes per second) or Gbps (giga bits per second)? I see mixed usage in this comment thread so I’m left wondering what it actually is.
The article is consistent and uses Gigabytes.
Re:How to Scale Your Model: How to Think About GPUs
by shaklee3 on Thursday August 21, 2025 @12:47AM
(#44969111)
GBps
Re:How to Scale Your Model: How to Think About GPUs
by nickysielicki on Thursday August 28, 2025 @08:06PM
(#45058434)
Try running a single node all to all with sharp disabled. I don’t believe you’ll see 450GB/s.
Quiz 2 is confusingly worded but is, iiuc, referring to intranode GPU connections rather than internode networking.