News for hackers. Stuff that matters.
Re:DiLoCo: Distributed Low-Communication Training of Languag... by lumost on Wednesday December 06, 2023 @09:36PM (#38552142)
The gradient is the direction and magnitude of change for each model weight. The optimizer determines how to adjust the weight based on the gradient.

There are no comments on this comment yet.