Re:Run LLMs at home, BitTorrent‑style
by Zetobal on Sunday September 17, 2023 @04:35PM
(#37549046)
An H100 is maybe a car but not nearly close to a house...
[ Parent
| on HN ]
Maybe not in your area, but it's very doable in other places, like where I live.
Re:Run LLMs at home, BitTorrent‑style
by ShamelessC on Sunday September 17, 2023 @08:54PM
(#37550964)
You expect me to believe there are other places than where I live?!
Re:Run LLMs at home, BitTorrent‑style
by nextaccountic on Sunday September 17, 2023 @05:23PM
(#37549481)
Is a single H100 enough?
Re:Run LLMs at home, BitTorrent‑style
by brucethemoose2 on Sunday September 17, 2023 @11:17PM
(#37551824)
80GB is enough, yeah.
I'm not sure what exact LORA/quantization settings would be ideal, but check out https://github.com/OpenAccess-AI-Collective/axolotl#config