Re:Run LLMs at home, BitTorrent‑style
by contravariant on Sunday September 17, 2023 @05:08PM
(#37549328)
In ordinary gradient descent the order does matter, since the position changes in between. I think stochastic gradient descent does sum a couple of gradients together sometimes, but I'm not sure what the trade-offs are and if LLMs do so as well.
[ Parent
| on HN ]
There are no comments on this comment yet.