News for hackers. Stuff that matters.

A comment on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance. Parent comment

Re:Mistral CEO confirms 'leak' of new open source AI model n... by speedgoose on Wednesday January 31, 2024 @04:35PM (#39209723)
Have you tried ollama? It’s another llama.cpp server implementation that is becoming popular.
[ Parent | on HN ]
Re:Mistral CEO confirms 'leak' of new open source AI model n... by gbickford on Wednesday January 31, 2024 @04:46PM (#39209855)
The authors don't seem to care about the principle of least privilege: https://github.com/ollama/ollama/issues/851#issuecomment-177...

It makes me wonder what other security issues they might now care about.

[ Parent | on HN ]
Re:Mistral CEO confirms 'leak' of new open source AI model n... by smoothjazz on Wednesday January 31, 2024 @04:57PM (#39209975)
This is a mac problem, not an ollama problem. It also sounds like it's solved by using homebrew (or linux).
[ Parent | on HN ]
Re:Mistral CEO confirms 'leak' of new open source AI model n... by brucethemoose2 on Wednesday January 31, 2024 @06:26PM (#39210937)
(afaik) it does not support batching, and some other server focused features.

I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.

[ Parent | on HN ]