Re:Bugs in LLM Training – Gradient Accumulation Fix
by danielhanchen on Friday October 18, 2024 @02:32PM
(#41882151)
Unfortunately it's not an Unsloth issue but a general issue affecting nearly all trainers which use grad accum. We worked with Huggingface so their trainers should be fixed now though in the main branch
[ Parent
| on HN ]
There are no comments on this comment yet.