News for hackers. Stuff that matters.
Re:A poor man's guide to fine-tuning Llama 2 by mfalcon on Tuesday September 26, 2023 @09:09AM (#37658739)
Hey man, thanks for the article. I like it that it is concise and simple. One thing that's not clear to me is about the inference stage: where does this inference process runs? Do you need to run it in a GPU powered instance or could it be runned in a consumer laptop?
Re:A poor man's guide to fine-tuning Llama 2 by cosmonoot on Tuesday September 26, 2023 @01:41PM (#37662959)
really appreciate the simple explanation
Re:A poor man's guide to fine-tuning Llama 2 by iterateAutomate on Tuesday September 26, 2023 @03:21PM (#37664512)
I'm curious to see the results, although as an english-only speaker, it may be hard for me to assess. Interesting write up nonetheless
Re:A poor man's guide to fine-tuning Llama 2 by neoneye2 on Tuesday September 26, 2023 @07:58PM (#37667655)
Has anyone run this on a Mac with M1 chip with GPU acceleration enabled?

I did my first hello world fine-tuning of Llama2 today, using Google Colab and various pieces of code. So no resume from checkpoint, I had to specify the number of epochs manually and no tracking of the loss and I didn't get the lora weights extracted, with llama2 7B I ended up with a 13gb gguf file, that I have successfully tried out locally with llama.cpp and its webui.

The axolotl seems like a nice project that hopefully makes the fine tuning easier.

Re:A poor man's guide to fine-tuning Llama 2 by ttt3ts on Wednesday September 27, 2023 @12:47AM (#37670285)
An M1 does not have the umph for training :( Just rent or buy a GPU. You can train a 7B model easily on a consumer GPU.