News for hackers. Stuff that matters.

A comment on Dalai: Automatically install, run, and play with LLaMA on your computer. Parent comment

Re:Dalai: Automatically install, run, and play with LLaMA on... by MacsHeadroom on Wednesday March 15, 2023 @05:05AM (#35165641)
Yes. Starting with the Facebook versions of LLaMA-7B you just quantize the model to 4bit on your desktop (since it takes 14GB of RAM) and then move it to your phone and follow the Android instructions in the repo. https://github.com/ggerganov/llama.cpp/#android

I've seen dozens of screenshots of it running in termux on androids by now at completely usable speeds.

[ Parent | on HN ]
Re:Dalai: Automatically install, run, and play with LLaMA on... by sucram1 on Wednesday March 15, 2023 @02:58PM (#35173091)
Thank you for the link! Insane that this can run on a phone.

As my current potato computer has 8GB of RAM, I'll ask a friend to do it :-)

[ Parent | on HN ]