It is not typically possible to blend models like that, since the training process is (lateral) order insensitive, as far as the model goes.
Re:Stable Diffusion 2.0 Release
by ShamelessC on Thursday November 24, 2022 @04:37AM
(#33729385)
Is that still the case when all models have a common ancestor (i.e. finetuned) and haven’t yet overfit on new data?
I thought so too until found that there are quite a bit of literatures nowadays about "merging" weights, for example, this one: https://arxiv.org/pdf/1811.10515.pdf and also the OpenCLIP paper.
Someone even figured out they could get great compression of specialized model files by first subtracting the base model from the specialized model (using plain arithmetic) before zipping it. Of course, you need the same base file handy when you go to reverse the process.