A simple guide to fine-tuning Llama 2

Re:A simple guide to fine-tuning Llama 2 by syntaxing on Monday July 24, 2023 @03:49PM (#36853313)

Can someone share a good tutorial how to prepare the data? And for fine tuning, does a 3090 have enough VRAM? I want to do what the author mentioned by fine tuning the model on my personal data but I’m not sure how to prepare the data. I tried using vector search + LLM but I find the results very subpar when using a local LLM.

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by jawerty on Monday July 24, 2023 @04:22PM (#36853686)

I just streamed this last night https://m.youtube.com/watch?v=TYgtG2Th6fI&t=3998s

I’ve been live streaming myself fine tuning llama on my GitHub data (to code like me)

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by jeremycarter on Monday July 24, 2023 @05:25PM (#36854411)

Fantastic job! Very easy to follow

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by jawerty on Monday July 24, 2023 @05:44PM (#36854646)

Thank you! I have some other streams where I do little projects like these check them out

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by samlhuillier on Monday July 24, 2023 @03:56PM (#36853393)

Working on this now!

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by syntaxing on Monday July 24, 2023 @04:06PM (#36853488)

I’m looking forward to this! Are you using an adapter (I don’t see it mentioned in your article)? I was under the impression you cannot fit 7B at 4 bit since it’ll take 25GB of VRAM or so.

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by samlhuillier on Monday July 24, 2023 @05:18PM (#36854328)

Yes using the qlora adapter that hugging face provides with peft

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by syntaxing on Monday July 24, 2023 @09:38PM (#36856763)

ahh, I was on my phone before so I must of glimpsed over it, I see it on the last section. Thanks!

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by notpublic on Monday July 24, 2023 @06:03PM (#36854887)

As mentioned in the OP's blog post, checkout https://github.com/facebookresearch/llama-recipes.git. specifically files in ft_datasets directory.

I am able to finetune meta-llama/Llama-2-13b-chat-hf on a 3090 using instructions from quickstart.ipynb.

[ Parent | on HN ]

Re:A simple guide to fine-tuning Llama 2 by syntaxing on Tuesday July 25, 2023 @07:42AM (#36860874)

Oh interesting, I didn’t know the documentation expanded so much in this past month.

[ Parent | on HN ]