I think part of the benefits of LoRA is that you can load the base model once, and then just swap out the vastly smaller LoRA fine-tune to fit the specific task it is working on.