Re:Mistral 7B Paper on ArXiv
by ramesh31 on Wednesday October 11, 2023 @10:27AM
(#37844817)
Is it better than llama 2?
[ Parent
| on HN ]
It is better than llama 2 7b and 13b. I tried the OpenOrca fine tune and it is very good, even when 4-bit quantized
What does OpenOrca do? It’s just instruction tuning it?
Yes, it is a instruction tune dataset: https://huggingface.co/datasets/Open-Orca/OpenOrca
It felt different from the official Mistral7B-Instruct. One of the highlights with the OpenOrca version is that you can steer the model with a system prompt (eg "You are a 5 year old")
Yes
Re:Mistral 7B Paper on ArXiv
by sebzim4500 on Wednesday October 11, 2023 @10:47AM
(#37845113)
For its size, yes. In absolute terms it is obviously less capable than llama-2-70B
For now. Huggingface[0] mentioned a DPO-fine-tuned version, Zephyr 7B, which it claims is competitive with Llama2-70B[1].
[0]: https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat
[1]: https://twitter.com/huggingface/status/1711780979574976661
Re:Mistral 7B Paper on ArXiv
by brucethemoose2 on Wednesday October 11, 2023 @03:58PM
(#37849499)
Oh, they uploaded the weights. I missed this one, cheers!