Yeah, that would be really interesting, open source models with similar quality to GPT that even smaller players could use to train models tailored for their application. Kind of the language equivalent to what the OS community already achieved for image models, eg the replicating Dreambooth for StableDiffusion. I think one problem standing in the way of that would be that the computational requirements for text just seem to be a lot higher.