Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Leaderboard

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by dmezzetti on Sunday December 03, 2023 @10:26AM (#38507715)

It's important to note that while the LLM Leaderboard (and the MTEB leaderboard) are good guides, being the top scoring model doesn't always mean much.

For example, in my projects, I've consistently found that the OpenOrca Mistral fine-tune (https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca) works better. Once again, that's just my experience and probably isn't universal. But it's important to try a couple models out to see if one that scores lower perhaps is better for your use case.

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by hskalin on Sunday December 03, 2023 @10:36AM (#38507797)

That is what my experience has been. I have tried OpenHermes2.5 (very hyped 7B model) vs basic Mistral 7B and tbh I prefer Mistral's answer for most things.

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by dmezzetti on Sunday December 03, 2023 @10:38AM (#38507823)

Exactly, benchmarks are just a guide. Same thing with vector models. Likely just a matter of how much your data overlaps with the fine-tuning dataset for a particular model.

[ Parent | on HN ]