News for hackers. Stuff that matters.

A comment on Making AMD GPUs competitive for LLM inference. Parent comment

Re:Making AMD GPUs competitive for LLM inference by cameron_b on Tuesday December 24, 2024 @12:07PM (#42503178)
I want to double-down on this statement, and call attention to the competitive nature of it. Specifically, I have recently tried to set up Triton on arm hardware. One might presume Nvidia would give attention to an architecture they develop, but the way forward is not easy. For some version of Ubuntu, you might have the correct version of python ( usually older than packaged ) but current LTS is out of luck for guidance or packages.

https://github.com/triton-lang/triton/issues/4978

[ Parent | on HN ]
Re:Making AMD GPUs competitive for LLM inference by claytonjy on Tuesday December 24, 2024 @07:16PM (#42505945)
I think you’ve mixed up your Triton’s; I’m talking about Triton Inference Server from NVIDIA while you’re talking about Triton the CUDA replacement from OpenAI
[ Parent | on HN ]