News for hackers. Stuff that matters.

A comment on Mistral ships le chat – enterprise AI assistant that can run on prem. Parent comment

Re:Mistral ships le chat – enterprise AI assistant that can ... by nicce on Wednesday May 07, 2025 @04:42PM (#43920364)
> Btw, you can also run Mistral locally within the Docker model runner on a Mac.

Efficiently? I thought macOS does not have API so that Docker could use GPU.

[ Parent | on HN ]
Re:Mistral ships le chat – enterprise AI assistant that can ... by jt_b on Wednesday May 07, 2025 @05:27PM (#43920740)
I haven't/wouldn't use it because I have a decent K8S ollama/open-webui setup, but docker announced this a month ago: https://www.docker.com/blog/introducing-docker-model-runner
[ Parent | on HN ]
Re:Mistral ships le chat – enterprise AI assistant that can ... by nicce on Wednesday May 07, 2025 @05:46PM (#43920880)
Hmm, I guess that is not actually running inside container/ there is no isolation. Some kind of new way that mixes llama.cpp , OCI format and docker CLI.
[ Parent | on HN ]