News for hackers. Stuff that matters.

A comment on Llama.cpp guide – Running LLMs locally on any hardware, from scratch. Parent comment

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by sorenjan on Friday November 29, 2024 @01:05PM (#42275703)
You can use Ollama for serving a model locally, and Continue to use it in VSCode.

https://ollama.com/blog/continue-code-assistant

[ Parent | on HN ]
Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by syntaxing on Friday November 29, 2024 @01:17PM (#42275803)
Relevant telemetry information. I didn’t like how they went from opt-in to opt-out earlier this year.

https://docs.continue.dev/telemetry

[ Parent | on HN ]
Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by homarp on Friday November 29, 2024 @06:16PM (#42277984)
you can do that with llama-server too
[ Parent | on HN ]
Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by freehorse on Saturday November 30, 2024 @03:29AM (#42280362)
Is autocomplete working well?
[ Parent | on HN ]