Re:The False Promise of Imitating Proprietary LLMs
by ThorsBane on Friday May 26, 2023 @08:31PM
(#36090992)
Any recommended starting points for LORA training llama 30B on a specific niche? Books, tutorials, videos are all appreciated. Thanks for your time!
Re:The False Promise of Imitating Proprietary LLMs
by f_devd on Friday May 26, 2023 @09:23PM
(#36091233)
Currently SOTA for specialization of LLMs is QLoRA: https://github.com/artidoro/qlora
But one great thing about open source LLMs is that you can specialize them in various tasks with affordable LORA training, enough to easily beat GPT4 in a specific niche.