News for hackers. Stuff that matters.

A comment on Dalai: Automatically install, run, and play with LLaMA on your computer. Parent comment

Re:Dalai: Automatically install, run, and play with LLaMA on... by skykooler on Sunday March 12, 2023 @08:28PM (#35128840)
How powerful of a computer does this need? It would be useful to see, for one thing, minimum RAM requirements for these models.
[ Parent | on HN ]
Re:Dalai: Automatically install, run, and play with LLaMA on... by spion on Monday March 13, 2023 @06:02AM (#35133497)
llama.cpp needs 40GB for the 65B model (due to int4 quantization)

RamNeeded(other_weight) = 40GB * other_weight/65B

[ Parent | on HN ]