News for hackers. Stuff that matters.

A comment on LoRA: Low-Rank Adaptation of Large Language Models. Parent comment

Re:LoRA: Low-Rank Adaptation of Large Language Models by seydor on Friday March 24, 2023 @01:33PM (#35292626)
Can rank decomposition be used to reduce the original weight matrices as well? Or are they assumed to be compressed already?
[ Parent | on HN ]
Re:LoRA: Low-Rank Adaptation of Large Language Models by metanonsense on Wednesday March 29, 2023 @03:33AM (#35353840)
Those fully trained networks are usually considered full-rank. At least that is what they say in the LoRA paper.
[ Parent | on HN ]