News for hackers. Stuff that matters.
Re:Intel's Immiseration – The Chip Letter by alecco on Friday August 09, 2024 @08:02AM (#41201001)
Gaudi was made by an Israeli company Intel acquired in 2019 (not an internal project). Gaudi 3 has PCIe 4.0 (vs. H100 PCIe 5.0, so 2x the bandwidth). It's strange for Intel, of all vendors, to lag behind in PCIe. And Nvidia has SXM for larger models (5x bandwidth).

"N5, PCIe 4.0, and HBM2e. This chip was probably delayed two years." -wmf

Re:Intel's Immiseration – The Chip Letter by ein0p on Friday August 09, 2024 @12:09PM (#41203152)
PCIe doesn't matter - these accelerators talk to one another, and Gaudi is outstanding in this regard. HBM2e also doesn't matter if you run decent sized batches (which you should, for throughput). In fact, HBM2e, being far less supply constrained, might even be an advantage.
Re:Intel's Immiseration – The Chip Letter by alecco on Sunday August 11, 2024 @09:31AM (#41216018)
Besides that it does matter (e.g. loading/saving LLMs), it speaks how Gaudi is not even using Intel's latest technology. They are clearly not in the main production pipeline. So Gaudi is not going to save Intel.
Re:Intel's Immiseration – The Chip Letter by ein0p on Sunday August 11, 2024 @02:23PM (#41218281)
But you don’t “load” models during inference. They are already on-device.