Dalai: Automatically install, run, and play with LLaMA on your computer

Re:Dalai: Automatically install, run, and play with LLaMA on... by GordonS on Monday March 13, 2023 @10:49AM (#35136445)

Nice, main.exe seems to work just fine with the 7B quantized model - generates a token every 400ms on an AMD Ryzen 5 2600!

But, quantize.exe doesn't seem to work - any valid command (such as below) pauses for a split second, then returns with no output?

$ quantize.exe ggml-model-f16.bin ggml-model-q4_0.bin 2

[ Parent | on HN ]

Re:Dalai: Automatically install, run, and play with LLaMA on... by GordonS on Tuesday March 14, 2023 @06:28AM (#35149454)

In case this helps anyone else: I built it myself on Windows with CMake, and then everything just works.

[ Parent | on HN ]

Re:Dalai: Automatically install, run, and play with LLaMA on... by starik36 on Wednesday March 15, 2023 @02:30PM (#35172640)

Do you mind sharing the binaries?

[ Parent | on HN ]

Re:Dalai: Automatically install, run, and play with LLaMA on... by GordonS on Thursday March 16, 2023 @06:27AM (#35180155)

Sure! https://filetransfer.io/data-package/8hxKAiaH#link

I wasn't sure where to upload them, and that link is only good for 50 downloads. Can put them somewhere else if you know a better location that doesn't require signup.

[ Parent | on HN ]

Re:Dalai: Automatically install, run, and play with LLaMA on... by starik36 on Thursday March 16, 2023 @12:09PM (#35183867)

Thank you.

llama.exe is basically main.exe?

I actually learned how to compile this code via CMake/VS2019. It's sure a whole lot more complicated then it was 25 years ago when I was writing C.

[ Parent | on HN ]

Re:Dalai: Automatically install, run, and play with LLaMA on... by GordonS on Friday March 17, 2023 @06:08AM (#35195128)

Yes, llama.exe is actually the name the project produces - the other poster must have renamed it to main.exe.

I just did `scoop install cmake`, then built from the command line, was a doddle!

[ Parent | on HN ]