News for hackers. Stuff that matters.

A comment on Mistral CEO confirms 'leak' of new open source AI model nearing GPT4 performance. Parent comment

Re:Mistral CEO confirms 'leak' of new open source AI model n... by declaredapple on Wednesday January 31, 2024 @04:55PM (#39209958)
Just to throw out there - I haven't had any issues with exllama personally, and it's a lot faster last I checked.
[ Parent | on HN ]
Re:Mistral CEO confirms 'leak' of new open source AI model n... by brucethemoose2 on Wednesday January 31, 2024 @06:46PM (#39211132)
Yeah this is what I use as well. The arbitary quantization is just great, as is everything else.

Its not easy to install though, and big dGPU only.

[ Parent | on HN ]