News for hackers. Stuff that matters.
Re:Run LLMs at home, BitTorrent‑style by contravariant on Sunday September 17, 2023 @05:08PM (#37549328)
In ordinary gradient descent the order does matter, since the position changes in between. I think stochastic gradient descent does sum a couple of gradients together sometimes, but I'm not sure what the trade-offs are and if LLMs do so as well.

There are no comments on this comment yet.