News for hackers. Stuff that matters.
Re:RedPajama: Reproduction of Llama with Friendly License (S... by nl on Monday April 17, 2023 @07:31PM (#35608041)
I'm not aware of any efficient transformer training code for AMD cards.

Also most training is done using bfloat, not single precision (which is usually only used for accumulators)

Re:RedPajama: Reproduction of Llama with Friendly License (S... by Robotbeat on Monday April 17, 2023 @09:42PM (#35608973)
Sure, you would need to rewrite the training code for AMD's ecosystem. If you're using mixed precision training, I suppose you're right about BF16. That puts the relative performance of A100 about 2.5x that of the Radeon RX 7900 XT. May be better to go with the Nvidia GeForce RTX 4090 with a $1600 retail.
Re:RedPajama: Reproduction of Llama with Friendly License (S... by titaniumtown on Tuesday April 18, 2023 @10:58AM (#35614854)
It all works with pytorch and huggingface's transformers library out of the box with Rocm.
Re:RedPajama: Reproduction of Llama with Friendly License (S... by slavik81 on Wednesday April 19, 2023 @01:27AM (#35624572)
You would need to compile a few components from source for Navi 31 if you were to try it today, so out-of-the-box is perhaps an overstatement, but it's certainly doable.