Re:DiLoCo: Distributed Low-Communication Training of Languag...
by lumost on Wednesday December 06, 2023 @09:36PM
(#38552142)
The gradient is the direction and magnitude of change for each model weight. The optimizer determines how to adjust the weight based on the gradient.
[ Parent
| on HN ]
There are no comments on this comment yet.