News for hackers. Stuff that matters.
Re:I Run LLMs Locally by d3VwsX on Monday December 30, 2024 @04:48PM (#42553926)
If I understand the distinction correctly, I run llamafile as a backend. I start it with the filename of a model on the command-line (might need a -M flag or something) and it will start up a chat-prompt for interaction in the terminal but also opens a port that speaks some protocol that I can connect to using a frontend (in my case usually gptel in emacs).

There are no comments on this comment yet.