This shouldn’t be surprising. Weights are in the training set and are necessarily structured. We should be asking the model to extrapolate on the structure.