News for hackers. Stuff that matters.

A comment on Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models. Parent comment

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by nacs on Friday August 11, 2023 @01:47PM (#37091586)
Absolutely. For QLORA / 4bit / GPTQ finetuning, you can train a 7B easily on an RTX 3060 (12GB VRAM).

If you have a 24GB VRAM GPU like a RTX 3090/4090, you can Qlora finetune a 13B or even a 30B model (in a few hours).

[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by kouroshh on Friday August 11, 2023 @02:02PM (#37091774)
Would be good to see a rigorous analysis of these PEFT methods on quality. There still seems to be a debate on whether these methods sacrifice quality or not.
[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by jawerty on Friday August 11, 2023 @01:50PM (#37091617)
+1 this
[ Parent | on HN ]