Re:LoRA: Low-Rank Adaptation of Large Language Models
by edwardjhu on Saturday March 25, 2023 @06:14PM
(#35308016)
Yes, the plan is to keep the original weights in VRAM and merge/unmerge LoRA weights on the fly. You can even cache a large library of LoRA ckpts in RAM.
Yup, I am!