News for hackers. Stuff that matters.

A comment on AMD's MI300X Outperforms Nvidia's H100 for LLM Inference. Parent comment

Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by epolanski on Thursday June 13, 2024 @05:30AM (#40667590)
Well, there's the beauty of specifying exactly how you ran your benchmark, it is easy to reproduce and disprove or confirm (assuming you got the hardware).
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by scotty79 on Thursday June 13, 2024 @07:47AM (#40668448)
As easy as getting yourself 8 H100 and 8 MI300X.

Fun weekend project for anybody.

[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by idiliv on Thursday June 13, 2024 @07:56AM (#40668538)
You can rent them online for ~ 4-5 $ per hour per GPU. Not cheap, but definitely feasible as a weekend project.
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by _zoltan_ on Thursday June 13, 2024 @08:32AM (#40668855)
where can I rent a H100 for 4-5 dollars an hour?

AWS doesn't let you use p5 instances (not getting a quota as a private person), lambda cloud is sold out.

[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by lhl on Thursday June 13, 2024 @08:52AM (#40669041)
It looks like Runpod currently (checked right now) has "Low" availability of 8x MI300 SXM (8x$4.89/h), H100 NVL (8x$4.39/h), and H100 (8x$4.69/h) nodes for anyone w/ some time to kill that wants to give the shootout a try.
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by darrick_horton on Thursday June 13, 2024 @10:26AM (#40670112)
We'd be happy to provide access to MI300X at TensorWave so you can validate our results! Just shoot us an email or fill out the form on our website
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by Jlagreen on Monday June 17, 2024 @10:24AM (#40705939)
If you're able to advertise available GPU compute in some public forums then it's enough to tell us about the demand of MI300X in cloud ...
[ Parent | on HN ]
Re:AMD's MI300X Outperforms Nvidia's H100 for LLM Inference by lhl on Tuesday June 18, 2024 @01:39AM (#40714377)
You're joking/trolling right? There are literally 10's of thousands of H100s available on gpulist right now, does that mean there's no cloud demand for Nvidia gpus? (I notice from your comment history that you seem to be some sort of bizarre NVDA stan account, but come on, be serious)
[ Parent | on HN ]