News for hackers. Stuff that matters.
Re:Making AMD GPUs competitive for LLM inference by 0xDEADFED5 on Tuesday December 24, 2024 @08:54PM (#42506314)
Fwiw on 22.04 i can use current kernel but otherwise follow Intel's instructions and the stuff works (old as it is now). I'm currently trying to figure out the best way to finetune Qwen 2.5 3B, the old axolotl ain't up to it. Not sure if I'm gonna work on a fork of axolotl or try something else at this point.

There are no comments on this comment yet.