News for hackers. Stuff that matters.

A comment on Dalai: Automatically install, run, and play with LLaMA on your computer. Parent comment

Re:Dalai: Automatically install, run, and play with LLaMA on... by yawnxyz on Sunday March 12, 2023 @06:34PM (#35127226)
Wow that's so incredible. Thanks for putting this together!

Do you have any machine specs associated with this? Can an old-ish Macbook Pro run this service?

I'm also curious, since I'm new to all this — is it possible to run something like this on Fly.io or does it take up way too much space?

[ Parent | on HN ]
Re:Dalai: Automatically install, run, and play with LLaMA on... by sp332 on Sunday March 12, 2023 @06:50PM (#35127493)
7B is the default. If it's quantized to 4 bits, that's a 3.9 GB file.
[ Parent | on HN ]