News for hackers. Stuff that matters.

A comment on Many options for running Mistral models in your terminal using LLM. Parent comment

Re:Many options for running Mistral models in your terminal ... by bugglebeetle on Tuesday December 19, 2023 @12:44PM (#38698785)
Unified memory and optimizations in llama.cpp (which Ollama wraps).
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by ithkuil on Tuesday December 19, 2023 @02:53PM (#38700624)
Is that using the GPU?
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by bugglebeetle on Tuesday December 19, 2023 @03:35PM (#38701137)
It can be variably configured. There are details in the repo, but llama.cpp makes use of Metal.
[ Parent | on HN ]