News for hackers. Stuff that matters.

A comment on Making AMD GPUs competitive for LLM inference. Parent comment

Re:Making AMD GPUs competitive for LLM inference by MrBuddyCasino on Tuesday December 24, 2024 @09:38AM (#42502168)
Fun fact: Nvidia H200 are currently half the price/hr of H100 bc people can’t get vLLM to work on it.

https://x.com/nisten/status/1871325538335486049

[ Parent | on HN ]
Re:Making AMD GPUs competitive for LLM inference by adrian_b on Tuesday December 24, 2024 @11:24AM (#42502903)
That seems like a CPU problem, not a GPU problem (due to Aarch64 replacing x86-64).
[ Parent | on HN ]
Re:Making AMD GPUs competitive for LLM inference by ryao on Tuesday December 24, 2024 @05:29PM (#42505409)
That is GH200 and it is likely due to an amd64 dependency in vLLM.
[ Parent | on HN ]