Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by SubiculumCode on Friday August 11, 2023 @01:36PM (#37091441)

one gpu? feasible with one 3060?

[ Parent | on HN ]

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by nacs on Friday August 11, 2023 @01:47PM (#37091586)

Absolutely. For QLORA / 4bit / GPTQ finetuning, you can train a 7B easily on an RTX 3060 (12GB VRAM).

If you have a 24GB VRAM GPU like a RTX 3090/4090, you can Qlora finetune a 13B or even a 30B model (in a few hours).

[ Parent | on HN ]

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by kouroshh on Friday August 11, 2023 @02:02PM (#37091774)

Would be good to see a rigorous analysis of these PEFT methods on quality. There still seems to be a debate on whether these methods sacrifice quality or not.

[ Parent | on HN ]

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by jawerty on Friday August 11, 2023 @01:50PM (#37091617)

+1 this

[ Parent | on HN ]