News for hackers. Stuff that matters.
Re:Mistral 7B,The complete Guide of the Best 7B model by avereveard on Tuesday October 31, 2023 @02:00AM (#38080841)
I've tested it and many fine tunes and while it seems to work ok for short chat it's really inattentive in summarisation and information extraction tasks. Continuation of long context stories/chat is also severely lacking, making up lot of contradictory details as it goes on.

It might or might not be better in benchmarks, but I just don't see it in practice for many tasks where you'd pick a LLM and go

There is one thing tho where I've seen it perform really well, which is single task fine tuning. It's small enough you can do full tuning and it's steerable enough you don't need more than few thousand samples.

There are no comments on this comment yet.