AMD has spent years developing MCM/multi-chip packaging to solve exactly this same problem. And they have Xilinx too, and Infinity Link (which is different from infinity fabric), and all these other things that help them build systems too. They aren't helpless babes unaware of the problem either.

(although I agree with the point that yes, NVIDIA was thinking about systems engineering much sooner and more comprehensively than AMD was. And not just multi-node scaling, but also things like the programming model (CUDA vs OpenCL/AMD APP/HSA Framework), and they also seem to have correctly identified that ethernet allows higher bandwidth-density than pcie as a transport for this interconnect, giving them an implementation advantage. Similarly, despite all the praise for AMD's packaging, it's often come with downsides and caveats, and NVIDIA usually isn't far behind with one that doesn't. Eg Fury X/Vega/Radeon VII vs GP100/GV100/GA100, or MI300X vs B200. And NVIDIA has explicitly been much earlier-to-market on the systems-scaling stuff like NVSwitch or pre-specced high-performance systems modules like DGX, which AMD still doesn't really have an answer for despite NVSwitch being over a decade old at this point etc.)

Tesla built a whole interconnect system for Dojo too, iirc, that was an explicit focus in their design. Not surprising for Jim Keller. But Jensen Huang nailed that requirement too.