Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5
by rbanffy on Friday December 19, 2025 @07:23AM
(#46325060)
Can’t you make bandwidth reservations and optimise data location to prefer comms between directly connected nodes over one or two-hop paths?
Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5
by KeplerBoy on Friday December 19, 2025 @07:47AM
(#46325222)
Sure, one could think of some kind of pipeline parallelism where you only need a fast transfer to the next step in the model and that would boost throughput but not increase model size.
Re:1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5
by Dylan16807 on Friday December 19, 2025 @12:53PM
(#46328748)
Obviously don't daisy chain, that wastes ports so badly. But if you connect 4 nodes into a loop, it goes fine. Relaying only adds 33% extra traffic. And what specifically are the latency numbers you have in mind?
If you have 3 links per box, then you can set up 8 nodes with a max distance of 2 hops and an average distance of 1.57 hops. That's not too bad. It's pretty close to having 2 links each to a big switch.
If you daisy chain four nodes, then traffic between nodes #1 and #4 eat up all of nodes #2 and #3's bandwidth, and you eat a big latency penalty. So, absent a switch, the fully connected mesh is the only way to have fast access to all the memory.