News for hackers. Stuff that matters.
Re:Mistral 7B,The complete Guide of the Best 7B model by behnamoh on Tuesday October 31, 2023 @01:13AM (#38080602)
IIRC there was a recent paper that showed Mistral may have data contamination, leading to high scores on the benchmarks. In my experiences with Zephyr, OpenOrca, and Dolphin finetunes of Mistral, the model did demonstrate ChatGPT-esque format in responses, but it didn't demonstrate any more intelligence than vanilla Llama 2 7B.

Another thing to worry about is that Mistral (the company) has publicly said that future larger models will not necessarily be open like the 7b model. After a certain model lock-in has taken place, the company will guard the model behind API (as they mentioned) and it won't be open anymore.

Re:Mistral 7B,The complete Guide of the Best 7B model by kcorbitt on Tuesday October 31, 2023 @01:22AM (#38080651)
I don't know whether Mistral had benchmark data contamination or not, but it's definitely a stronger model than Llama 2 7b or 13b even on non-benchmark tasks.

We proactively tested Mistral across all of our customer-deployed models on our fine-tuning platform (openpipe.ai) and found that with the exact same training data it outperformed both of those models consistently. At this point we're only keeping Llama 2 7b/13b around for our legacy customers and recommend that everyone training new models use Mistral because it's just so much better.

Re:Mistral 7B,The complete Guide of the Best 7B model by avereveard on Tuesday October 31, 2023 @02:00AM (#38080841)
I've tested it and many fine tunes and while it seems to work ok for short chat it's really inattentive in summarisation and information extraction tasks. Continuation of long context stories/chat is also severely lacking, making up lot of contradictory details as it goes on.

It might or might not be better in benchmarks, but I just don't see it in practice for many tasks where you'd pick a LLM and go

There is one thing tho where I've seen it perform really well, which is single task fine tuning. It's small enough you can do full tuning and it's steerable enough you don't need more than few thousand samples.

Re:Mistral 7B,The complete Guide of the Best 7B model by brucethemoose2 on Tuesday October 31, 2023 @02:15AM (#38080936)
Eh, I've found llama 7b finetunes to be underwhelming too. They both seem to fall apart at detail retrieval and such, even with Mistral's longer base context.

For what its worth, Mistral finetunes excel in this ranking, and Mistral AI almost certainly did not contaminate it with that test: https://rentry.co/ayumi_erp_rating

Re:Mistral 7B,The complete Guide of the Best 7B model by tayo42 on Tuesday October 31, 2023 @02:45AM (#38081055)
I wonder if it's possible to do something like folding at home but train an llm and have it always be open
Re:Mistral 7B,The complete Guide of the Best 7B model by brucethemoose2 on Tuesday October 31, 2023 @12:22PM (#38087262)
Re:Mistral 7B,The complete Guide of the Best 7B model by tayo42 on Tuesday October 31, 2023 @01:02PM (#38087961)
wow didnt know that, thats really cool, ill have to play with it
Re:Mistral 7B,The complete Guide of the Best 7B model by brucethemoose2 on Tuesday October 31, 2023 @02:09PM (#38088970)
If you want to host for the AI Horde, Aphrodite has the most GPU throughput, and Koboldcpp is the easiest and quite flexible:

https://github.com/PygmalionAI/aphrodite-engine https://github.com/LostRuins/koboldcpp

Picking a model to host is quite a rabbit hole, but I am very interested in SciPhi and Zephyr at the moment.

I havent messed with Petals yet either, but swarm finetuning is very appealing... If thats even implemented atm.