Re:LoRA from Scratch implementation for LLM finetuning
by wfalcon on Monday January 22, 2024 @05:32PM
(#39096466)
this is also what our (Lightning AI) lit-gpt library does.
https://github.com/Lightning-AI/lit-gpt
Re:LoRA from Scratch implementation for LLM finetuning
by jamesblonde on Tuesday January 23, 2024 @01:09AM
(#39100135)
Thanks, hadn't seen this.
I concentrate on collecting and curating the fine-tuning data, do "data-centric" fine-tuning - not learning LoRA from scratch.