Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Continue with LocalAI: An alternative to GitHub's Copilot that runs locally. Parent comment
./server -t 8 -m models/wizardcoder-python-34b-v1.0.Q4_K_S.gguf -c 16384 --mlock -ngl 60
[Edit: make sure to compile llama.cpp with GPU support first, e.g., "make clean && LLAMA_CUBLAS=1 make -j"]
There are no comments on this comment yet.
[Edit: make sure to compile llama.cpp with GPU support first, e.g., "make clean && LLAMA_CUBLAS=1 make -j"]