News for hackers. Stuff that matters.

A comment on Mistral AI Launches New 8x22B MOE Model. Parent comment

Re:Mistral AI Launches New 8x22B MOE Model by Jackson__ on Wednesday April 10, 2024 @02:02AM (#39987419)
My personal speculation is that their closed models are based on other companies' models.

For example on EQbench[0], Miqu[1], a leaked continued pretrain based on LLama2, performs extremely similar to the mistral medium model their API offers.

Maybe they're thinking it'd be bad PR for them to release models they didn't create from scratch, or there is some contractual obligation preventing the release.

[0]https://eqbench.com/index.html

[1]https://huggingface.co/miqudev/miqu-1-70b

[ Parent | on HN ]
Re:Mistral AI Launches New 8x22B MOE Model by moffkalast on Wednesday April 10, 2024 @07:22AM (#39989420)
That's quite likely, some have also speculated that Mistral 7B got some EU grant funding that stipulated it had to be openly released later, and Mixtral is based on Mistral 7B so it would likely be subject to the same terms. I haven't found any source to substantiate it though.
[ Parent | on HN ]