News for hackers. Stuff that matters.

A comment on Llamafile lets you distribute and run LLMs with a single file. Parent comment

Re:Llamafile lets you distribute and run LLMs with a single ... by simonw on Wednesday November 29, 2023 @08:42PM (#38468147)
Sounds like you should download the 4.45MB llamafile-server-0.1 executable from https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 and then run it against your existing gguf model files like this:

    ./llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf
See here: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-t...
[ Parent | on HN ]

There are no comments on this comment yet.