News for hackers. Stuff that matters.
Re:Run LLMs at home, BitTorrent‑style by whimsicalism on Sunday September 17, 2023 @05:37PM (#37549599)
By the “delta in the LLM weights”, I am assuming you mean the gradients. You are effectively describing large batch training (data parallelism) which is part of the way you can scale up but there are quickly diminishing returns to large batch sizes.

There are no comments on this comment yet.