LoRA: Low-Rank Adaptation of Large Language Models

Re:LoRA: Low-Rank Adaptation of Large Language Models by edwardjhu on Friday March 24, 2023 @04:01PM (#35294656)

> Merged means you are modifying the model weights, which means you are stuck with that one model on that device (though, this usually applies for most implementations for the unmerged versions too).

If one is careful with floating point issues, it's straightforward to unmerge the weights.

W_0 = W_1 - BA

Yes, prompt-based methods don't involve swapping weights.

[ Parent | on HN ]

Re:LoRA: Low-Rank Adaptation of Large Language Models by arugulum on Friday March 24, 2023 @05:13PM (#35295610)

Right, it's mathematically easy (again, up to floating point issues) to recover the weights as needed, but in terms of distribution/serving I'm guessing the plan is to have the original weights and carry around the LoRA weights and merge as necessary.

(Also, I'm assuming you're the first author of LoRA.)

[ Parent | on HN ]

Re:LoRA: Low-Rank Adaptation of Large Language Models by edwardjhu on Saturday March 25, 2023 @06:14PM (#35308016)

Yes, the plan is to keep the original weights in VRAM and merge/unmerge LoRA weights on the fly. You can even cache a large library of LoRA ckpts in RAM.

Yup, I am!

[ Parent | on HN ]