Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
by mistymountains on Thursday June 13, 2024 @11:25AM
(#40670818)
These kinds of comments make me think few people have actually tried. My experience has been 1 work day of getting things set up to work the same as before for training and testing (PyTorch).
[ Parent
| on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
by imtringued on Thursday June 13, 2024 @01:14PM
(#40672163)
You have to consider that the average person who tried to do machine learning on AMD GPUs got burned in the past decade and has no reason to change their opinion. Also in the past it was much harder to get access to cutting edge GPUs from AMD. The fact that AMD drops GPU support for ROCm quickly also earns them scorn. I don't think it is an unfair assessment. They earned their reputation.
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
by muxr on Sunday June 16, 2024 @06:47PM
(#40700743)
ROCm has improved a lot. And you can rent mi300x in the cloud now. So if you have a program that runs on Nvidia GPUs, it takes no time to test it on a cloud mi300x. If it works you can use it and save some money in the process.