News for hackers. Stuff that matters.
Re:Mistral CEO confirms 'leak' of new open source AI model n... by brucethemoose2 on Wednesday January 31, 2024 @06:26PM (#39210937)
(afaik) it does not support batching, and some other server focused features.

I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.

There are no comments on this comment yet.