News for hackers. Stuff that matters.
Re:LoRA: Low-Rank Adaptation of Large Language Models by edwardjhu on Saturday March 25, 2023 @06:11PM (#35307993)
If latency is irrelevant, I don't think there is a strong practical reason to prefer one over another. (LoRA is more elegant in my biased opinion because you roughly recover finetuning with a large r.) In practice, you see one do a little better on some tasks and vice versa on others as observed by papers after mine.

There are no comments on this comment yet.