News for hackers. Stuff that matters.

A comment on Dalai: Automatically install, run, and play with LLaMA on your computer. Parent comment

Re:Dalai: Automatically install, run, and play with LLaMA on... by hbbio on Sunday March 12, 2023 @08:44PM (#35128945)
Happened to me as well. Apparently, you can just run:

   python3.10 convert-pth-to-ggml.py models/7B 1
   ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
And then play with:

    ./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128 -p "..."
[ Parent | on HN ]

There are no comments on this comment yet.