Re:Fine tune LLAMA3 on million scale dataset in consumer GPU...
by bradfox2 on Saturday May 04, 2024 @11:27PM
(#40262053)
I don't have the arxiv link bookmarked, but there was a paper written on pretraining with Lora. It involved merging adapters back every n steps with good results.
[ Parent
| on HN ]
There are no comments on this comment yet.