News for hackers. Stuff that matters.
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by preisschild on Thursday June 13, 2024 @10:23AM (#40670066)
But Apple doesn't produce servers or server hardware.
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by bbatha on Thursday June 13, 2024 @12:09PM (#40671415)
Currently no, but the XServe was a product for a decade. And they have built an internal ML cloud, presumably with rack mountable hardware. The bigger issue for apple IMO is they ditched the server features of their OS and they're not going to sell a hypothetical M4ultra Xserve with linux.
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by threecheese on Thursday June 13, 2024 @10:42AM (#40670318)
Good point; if the Mx architecture does prove to be a viable competitor to Nvidia/AMD for training and/or inference, do you think Apple would enter the server market? They continue to diversify on the consumer side, I wonder if they have their eye on the business market; I am not sure if their general strategy of “prettier is better” would work well there though.
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by sroussey on Thursday June 13, 2024 @11:17AM (#40670745)
They should buy Grok (the hardware inference company, not the lame twitter bot).
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by ein0p on Thursday June 13, 2024 @01:46PM (#40672490)
I don’t think that’s true anymore: see WWDC presentation about Private Cloud
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by fmajid on Saturday June 15, 2024 @09:15AM (#40689595)
Clearly they are building their own Apple Silicon powered servers for their Private Computing Cloud, even if it is not sold to outsiders like the XServe used to be.