News for hackers. Stuff that matters.

A comment on AMD's MI300X Outperforms Nvidia's H100 for LLM Inference. Parent comment

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by sigmoid10 on Sunday June 16, 2024 @05:09AM (#40695710)
Even then it's an insanely bold assumption that a company other than Nvidia could build a better framework than CUDA for compiling PTX. Especially since CUDA is already so much C-like. I've never seen anyone go deeper than that outside of academia.
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by dexterac on Monday June 17, 2024 @01:27AM (#40702566)
How many customers/consumers will care about services are be built with CUDA?

If they need a ChatBot that uses a model with same accuracy and performance as on non-CUDA hardware, would they still want CUDA based hardware?

[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by sigmoid10 on Saturday June 22, 2024 @05:29AM (#40757640)
Who is going to build the architecture and compile the device specific kernels? You have to pay those people as well and you can save tons of money and time if you do it with cuda.
[ Parent | on HN ]