Llamafile lets you distribute and run LLMs with a single file

Re:Llamafile lets you distribute and run LLMs with a single ... by zoe_dk on Thursday November 30, 2023 @01:27AM (#38470352)

Noob question - how might I call this from my Python script? Say as a replacement gpt3.5 turbo of sorts. Is there an option without GUI?

This is great thank you, very user friendly (exhibit a: me)

[ Parent | on HN ]

Re:Llamafile lets you distribute and run LLMs with a single ... by simonw on Thursday November 30, 2023 @07:56AM (#38472893)

The llama.cpp server version runs a JSON API that you can call. It's currently missing any documentation though as far as I can tell - I found dome details on Reddit: https://www.reddit.com/r/LocalLLaMA/comments/185kbtg/llamacp...

[ Parent | on HN ]

Re:Llamafile lets you distribute and run LLMs with a single ... by zoe_dk on Thursday November 30, 2023 @01:31PM (#38477149)

Thank you Simon I will check it out.

[ Parent | on HN ]