News for hackers. Stuff that matters.
Re:Bugs in LLM Training – Gradient Accumulation Fix by danielhanchen on Friday October 18, 2024 @02:32PM (#41882151)
Unfortunately it's not an Unsloth issue but a general issue affecting nearly all trainers which use grad accum. We worked with Huggingface so their trainers should be fixed now though in the main branch

There are no comments on this comment yet.