News for hackers. Stuff that matters.

A comment on Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook. Parent comment

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by visarga on Thursday March 16, 2023 @03:29PM (#35187070)
Just for quick reference:

LLM - the model

weights - the data of the model itself

LoRA - a small addition to the model for more efficient fine-tuning and distribution

[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by EGreg on Thursday March 16, 2023 @03:53PM (#35187445)
Tell us more about LoRA
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by visarga on Thursday March 16, 2023 @04:34PM (#35188109)
Instead of a matrix NN they use two matrices, Nd and d*N, and assuming d << N it would be like a lightweight addition. Keeping the original matrix unchanged, this new addition is simply added in parallel and initialised to low values.
[ Parent | on HN ]