ChatGPT Has Gone Berserk

Re:ChatGPT Has Gone Berserk by mark_l_watson on Wednesday February 21, 2024 @09:06AM (#39453954)

Mixtral 8x7b continues to amaze me, even though I have to run it with 3 bit quantization on my Mac (I just have 32G memory). When I run this model on commercial services with 4 or more bits of quantization I definitely notice, subjectively, better results.

I like to play around with smaller models and regular app code in Common Lisp or Racket, and Mistral 7b is very good for that. Mixing and matching old fashioned coding with the NLP, limited world knowledge, and data manipulation capabilities of LLMs.

[ Parent | on HN ]

Re:ChatGPT Has Gone Berserk by throwawaybbq1 on Wednesday February 21, 2024 @10:18PM (#39462787)

This is neat to know. On Ollama, I see mistral and mixtral. Is the latter one the MoE model?

[ Parent | on HN ]

Re:ChatGPT Has Gone Berserk by dkarras on Wednesday February 21, 2024 @11:01PM (#39463113)

yes, mixtral is the MoE model.

[ Parent | on HN ]

Re:ChatGPT Has Gone Berserk by dkarras on Wednesday February 21, 2024 @11:02PM (#39463119)

There is also MiQu (stands for mi(s|x)tral quantized I think?) which is a leaked and older mistral medium model. I have not been able to try it as it needs some RAM / VRAM I don't have but people say it is very good.

[ Parent | on HN ]