AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by DrNosferatu on Thursday June 13, 2024 @06:58AM (#40668067)

The comparison is between setups with different amounts of GPU RAM and there's no quantification of final performance/price.

[ Parent | on HN ]

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by Gasp0de on Thursday June 13, 2024 @07:17AM (#40668192)

So? If you get twice the RAM at a comparable price and that leads to twice the performance, what's wrong with comparing that?

[ Parent | on HN ]

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by DrNosferatu on Thursday June 13, 2024 @07:54AM (#40668518)

Nothing wrong - just for transparency.

Also, the price difference is not quantified.

Additionally, CUDA is a known and tangible software stack - can I try out this "MK1 FLywheel" on my local (AMD) hardware?

[ Parent | on HN ]

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by muxr on Sunday June 16, 2024 @06:49PM (#40700752)

None of those things matter, if all you're looking to do is run your existing workloads on mi300x in the cloud. You get more bang per buck by going AMD.

[ Parent | on HN ]