No mounting is perfectly rigid and non-deformable.

Thermal expansion was a consideration when I took an optomechanical design course 15 years ago, for example (we had to memorize expansion coefficients of various common materials for quizzes, much to my annoyance). Like the glass itself changes size, altering how it focuses without even getting into the housing, which can also change size, shift around, vibrate, etc. Optical systems need pretty precise alignment.