Re:Run LLMs at home, BitTorrent‑style
by nextaccountic on Sunday September 17, 2023 @05:23PM
(#37549481)
Is a single H100 enough?
[ Parent
| on HN ]
Re:Run LLMs at home, BitTorrent‑style
by brucethemoose2 on Sunday September 17, 2023 @11:17PM
(#37551824)
80GB is enough, yeah.
I'm not sure what exact LORA/quantization settings would be ideal, but check out https://github.com/OpenAccess-AI-Collective/axolotl#config