Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by Tostino on Monday June 26, 2023 @11:10AM (#36480648)

Look at QLoRA. The QLoRA can be attached to all layers, allowing you to alter behavior with much less data than the original LoRA implementation. It seems to "stick" better.

I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.

[ Parent | on HN ]

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by throwaway72762 on Monday June 26, 2023 @12:30PM (#36481972)

Are there any repos and steps you can point to to do this? I'd love to try to do exactly what you describe. I have been trying to do the same and have run into a lot of repos with broken dependencies.

[ Parent | on HN ]

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by Tostino on Monday June 26, 2023 @04:30PM (#36485407)

I used: https://github.com/artidoro/qlora but there are quite a few others that likely work better. It was literally my first attempt at doing anything like this, and took the better part of an evening to work through CUDA/Python issues to get it training, and ~20 hours of training.

[ Parent | on HN ]