News for hackers. Stuff that matters.
Re:Bugs in LLM Training – Gradient Accumulation Fix by danielhanchen on Friday October 18, 2024 @02:34PM (#41882181)
Unfortunately transformers is a general library for many models, and so there are tonnes of different architectures. Unfortunately copy paste and changing some parts of the arch is the only way feasible in the meantime.

There are no comments on this comment yet.