Re:Run LLMs at home, BitTorrent‑style
by whimsicalism on Sunday September 17, 2023 @05:37PM
(#37549599)
By the “delta in the LLM weights”, I am assuming you mean the gradients. You are effectively describing large batch training (data parallelism) which is part of the way you can scale up but there are quickly diminishing returns to large batch sizes.
[ Parent
| on HN ]
There are no comments on this comment yet.