News for hackers. Stuff that matters.

A comment on AMD's MI300X Outperforms Nvidia's H100 for LLM Inference. Parent comment

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by huevosabio on Thursday June 13, 2024 @05:30AM (#40667592)
RunPod [0] is pricing MI300X at $4.89/hr vs $3.89-4.69/hr for H100s.

So, probably around the same price?

The tests look promising, though!

[0] https://runpod.io/

[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by latchkey on Sunday June 16, 2024 @10:25PM (#40701820)
We are starting at $4.50/hr [0]. The catch is that we won't have availability until mid August.

The weird thing on Runpod is the virtual CPUs, you can't run MI300x in virtual machines yet. It is a missing feature that AMD is working on.

[0] https://hotaisle.xyz/pricing/

[ Parent | on HN ]