News for hackers. Stuff that matters.

A comment on LoRA: Low-Rank Adaptation of Large Language Models. Parent comment

Re:LoRA: Low-Rank Adaptation of Large Language Models by MacsHeadroom on Sunday March 26, 2023 @12:16PM (#35316145)
Modern peft methods with LoRA actually do reduce training time by orders of magnitude.

Here's an example of 20 seconds per epoch on a single consumer GPU: https://github.com/johnsmith0031/alpaca_lora_4bit/issues/7#i...

[ Parent | on HN ]

There are no comments on this comment yet.