1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5 by angoragoats on Thursday December 18, 2025 @08:08PM (#46321101)

The OP makes reference to this with a link to a GitHub repo that has some benchmarks. TCP over Thunderbolt compared to RDMA over Thunderbolt has roughly 7-10x higher latency, ~300us vs 30-50us. I would expect TCP over 200GbE to have similar latency to TCP over Thunderbolt.

Put another way, see the graphs in the OP where he points out that the old way of clustering performs worse the more machines you add? I’d expect that to happen with 200GbE also.

And with a switch, it would likely be even worse, since the hop to the switch adds additional latency that isn’t a factor in the TB5 setup.

[ Parent | on HN ]

Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5 by wmf on Thursday December 18, 2025 @09:14PM (#46321575)

You're ignoring RoCE which would have the same or lower latency than RoTB. And I think macOS already supports RoCE.

[ Parent | on HN ]

Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5 by angoragoats on Friday December 19, 2025 @07:09AM (#46324941)

MacOS does not support RoCE.

[ Parent | on HN ]

Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5 by Hikikomori on Friday December 19, 2025 @06:12AM (#46324522)

Switch probably does cut through so it starts forwarding the frame before its even fully received.

[ Parent | on HN ]