News for hackers. Stuff that matters.
Re:Run LLMs at home, BitTorrent‑style by brucethemoose2 on Sunday September 17, 2023 @11:17PM (#37551824)
80GB is enough, yeah.

I'm not sure what exact LORA/quantization settings would be ideal, but check out https://github.com/OpenAccess-AI-Collective/axolotl#config

There are no comments on this comment yet.