News for hackers. Stuff that matters.

A comment on Mistral "Mixtral" 8x7B 32k model [magnet]. Parent comment

Re:Mistral "Mixtral" 8x7B 32k model [magnet] by ukuina on Saturday December 09, 2023 @02:51AM (#38579671)
Possibly a huge leap forward in open-source model capability. GPT4's prowess supposedly comes from strong dataset + RLHF + MoE (Mixture of Experts).

Mixtral brings MoE to an already-powerful model.

[ Parent | on HN ]

There are no comments on this comment yet.