News for hackers. Stuff that matters.

A comment on We fine-tuned Llama 405B on AMD GPUs. Parent comment

Re:We fine-tuned Llama 405B on AMD GPUs by gdiamos on Tuesday September 24, 2024 @03:34AM (#41634049)
backward activations does but typically not backwards weight gradients.

Why compute gradients with regards to weights that aren't going to be updated?

[ Parent | on HN ]

There are no comments on this comment yet.