News for hackers. Stuff that matters.

A comment on Performance of llama.cpp on Apple Silicon A-series. Parent comment

Re:Performance of llama.cpp on Apple Silicon A-series by zakki on Tuesday December 19, 2023 @06:43PM (#38703567)
How do you use 8x7B?
[ Parent | on HN ]
Re:Performance of llama.cpp on Apple Silicon A-series by localhost on Tuesday December 19, 2023 @07:11PM (#38703795)
Use ollama [1]

$ ollama run mixtral

[1] https://github.com/jmorganca/ollama

[ Parent | on HN ]
Re:Performance of llama.cpp on Apple Silicon A-series by yieldcrv on Tuesday December 19, 2023 @06:59PM (#38703711)
running it with LM Studio. No command line and has a huggingface browser, very user friendly.
[ Parent | on HN ]