News for hackers. Stuff that matters.
Re:LoRA: Low-Rank Adaptation of Large Language Models by grph123dot on Friday March 24, 2023 @11:50AM (#35290729)
Since we are in ELI5, it seems that the concept of low rank approximation is required to understand this method.

(1) https://en.wikipedia.org/wiki/Low-rank_approximation

Edited: By the way, it seems to me that there is an error in the wikipedia page because if the Low-rank approximation takes a larger rank then the bound of the error should decrease, and in this page the error increases.

There are no comments on this comment yet.