I guess when it comes to LLM's what is considered the "source" - the weights or the code used to build the weights?

To the extent that the concept is applicable, it would be the training data and the training code.

You're forgetting the flour in your cake recipie, the data, arguably the singularly most important part.