We fine-tuned Llama 405B on AMD GPUs

Re:We fine-tuned Llama 405B on AMD GPUs by anthonix1 on Monday September 23, 2024 @10:37PM (#41632620)

I just asked an instance of Hermes 3 Llama 3.1 405B running on AMD GPUs: "does pytorch scaled dot product attention run on AMD GPUs?":

"Yes, PyTorch's scaled dot product attention can run on AMD GPUs. PyTorch supports AMD GPUs through the ROCm (Radeon Open Compute) platform. To use PyTorch with an AMD GPU, you need to install the ROCm version of PyTorch, which is specifically built for AMD GPUs [...]"

And it proceeded to give the steps to follow to install and run, with example Python code to demonstrate it. One slight nitpick is that it referred to an older URL with the --index-url to install torch with pip, but otherwise it was correct.

[ Parent | on HN ]

Re:We fine-tuned Llama 405B on AMD GPUs by unlikelymordant on Tuesday September 24, 2024 @02:17AM (#41633572)

are you at all confident that this isn't hallucinated? I'd never trust an answer like this from an LLM

[ Parent | on HN ]

Re:We fine-tuned Llama 405B on AMD GPUs by WithinReason on Tuesday September 24, 2024 @02:42AM (#41633690)

Did you verify everything else it said is true?

[ Parent | on HN ]