Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style...
by EGreg on Thursday March 16, 2023 @03:53PM
(#35187445)
Tell us more about LoRA
[ Parent
| on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style...
by visarga on Thursday March 16, 2023 @04:34PM
(#35188109)
Instead of a matrix NN they use two matrices, Nd and d*N, and assuming d << N it would be like a lightweight addition. Keeping the original matrix unchanged, this new addition is simply added in parallel and initialised to low values.