News for hackers. Stuff that matters.

A comment on LoRA: Low-Rank Adaptation of Large Language Models. Parent comment

Re:LoRA: Low-Rank Adaptation of Large Language Models by esperent on Friday March 24, 2023 @11:05PM (#35298864)
> about the same result

Can you qualify this? Is it still useful or not?

[ Parent | on HN ]
Re:LoRA: Low-Rank Adaptation of Large Language Models by MacsHeadroom on Sunday March 26, 2023 @12:22PM (#35316224)
>Is it still useful or not?

Is what still useful? A LoRA is about as good and useful as a full fine tune. If you have unlimited storage space to store them or unlimited compute to make them then I would still prefer full fine tunes. But the difference is marginal and generally not worth the storage space or increased compute costs for individuals.

[ Parent | on HN ]