News for hackers. Stuff that matters.

A comment on The Coming of Local LLMs. Parent comment

Re:The Coming of Local LLMs by qumpis on Tuesday April 11, 2023 @12:02PM (#35527075)
Inference, even fine-tuning a few layers would be difficult since one needs to use non-quantized model, I'd imagine
[ Parent | on HN ]
Re:The Coming of Local LLMs by ctoth on Tuesday April 11, 2023 @12:33PM (#35527518)
Checkout LoRA and Alpaca LoRA and the whole huge group of people who have already figured this out. I think there was another breakthrough (yesterday?) which is a further adaption of LoRA to touch even less parameters at runtime.
[ Parent | on HN ]
Re:The Coming of Local LLMs by homarp on Tuesday April 11, 2023 @12:45PM (#35527689)
DyLoRA https://news.ycombinator.com/item?id=35514228 is what you meant
[ Parent | on HN ]
Re:The Coming of Local LLMs by whimsicalism on Tuesday April 11, 2023 @12:55PM (#35527837)
Disagree that this is a major breakthrough :) but it is probably a marginal improvement.
[ Parent | on HN ]
Re:The Coming of Local LLMs by whimsicalism on Tuesday April 11, 2023 @12:55PM (#35527838)
They're not training on edge :)
[ Parent | on HN ]