If model weights have a logical and coherent structure, I don't think it would be surprising for models to have some understanding of them, the same they can understand language and code and images, given enough data.

What would be surprising is if this happened without any specific training on model experiments and interpretability.