Re:Ask HN: Has anyone replaced Claude/GPT with a local model...
by cpburns2009 on Wednesday June 17, 2026 @03:03PM
(#48575168)
My LLM host is a Framework Desktop 128GB (Ryzen AI 395) with an RTX 5090 (Minisforum DEG1 dock, cheap Oculink card plugged into the PCIe 4.0 x4 slot). I run llama-server (b9296 compiled for CUDA) using a systemd service, and its configured as:
I use the model from a sandboxed OpenCode using the @ai-sdk/openai-compatible plugin. And I'm using an SSH tunnel from home to work to expose llama-server to OpenCode.
[ Parent
| on HN ]
There are no comments on this comment yet.