I think it's because what constitutes core physics is more standardised and less variables across universities. In addition, the core syllabus of physics tends to have concepts that build on the previous very nicely and ultimately combine. In constrast, I think other subjects tend to specialise more quickly than physics.