News for hackers. Stuff that matters.
Re:DyLoRA: Parameter Efficient Tuning of Pre-Trained Models by lxe on Monday April 10, 2023 @03:22PM (#35516252)
Kudos for the authors for providing the code https://github.com/huawei-noah/KD-NLP/tree/main/DyLoRA and the roberta example. Considering the current state of the OSS LLM community, I'm guessing someone is already porting it to Llama and gpt-style models.
Re:DyLoRA: Parameter Efficient Tuning of Pre-Trained Models by kernelsanderz on Monday April 10, 2023 @06:21PM (#35518097)
Adding this to the huggingface peft library would be amazing. That's the main library that people using LoRA are currently using. https://github.com/huggingface/peft/issues/289
Re:DyLoRA: Parameter Efficient Tuning of Pre-Trained Models by brucethemoose2 on Monday April 10, 2023 @10:15PM (#35519909)
The stable diffusion community has, unfortunately, largely ignored peft because the training/inference scripts largely ignored diffusers.