News for hackers. Stuff that matters.
Re:The Coming of Local LLMs by boppo1 on Wednesday April 12, 2023 @12:27AM (#35535005)
30B llama needs a 3090 or 4090. 13B I think you can get away with a 3/4080. If you have 64 gigs of ram and a beefy CPU you can run even 65B, but boy it's slow.

13B is pretty meh, but 30B is great, if not quite Chatgpt. But I can ask it why my highschool geometry teacher was such a cunt and it will happily discuss the matter without reservation. Very therapeutic.

There are no comments on this comment yet.