News for hackers. Stuff that matters.

A comment on Mistral 7B, the most powerful language model for its size to date, Apache 2.0. Parent comment

Re:Mistral 7B, the most powerful language model for its size... by Patrick_Devine on Wednesday September 27, 2023 @03:12PM (#37679727)
I'm getting >30 tokens/sec using it with ollama and an M2 Pro. That might be a little slow though because I have a background finetuning job running.
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by minzi on Wednesday September 27, 2023 @06:45PM (#37682463)
Bit of a tangential question here, but any recommendations on how to get started fine tuning this model (or ones like it)? I feel like there are a million different tutorial and ways of doing it when I google.
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by Karrot_Kream on Wednesday September 27, 2023 @08:09PM (#37683397)
Take a look at the QLoRA repo https://github.com/artidoro/qlora/ which has an example finetuning Llama. Made by the authors of the QLoRA paper.
[ Parent | on HN ]
Re:Mistral 7B, the most powerful language model for its size... by brucethemoose2 on Wednesday September 27, 2023 @09:17PM (#37683950)
https://github.com/OpenAccess-AI-Collective/axolotl

This is a wrapper around many training methods, and it has yielded many excellent community finetunes already.

[ Parent | on HN ]