You don't even need to use exotic materials with negative thermal expansion: instead, you can use a combination of materials with different coefficients of thermal expansion pointed in opposite directions. This was one of the innovations that John Harrison made that allowed him to make precision timepieces.

https://en.wikipedia.org/wiki/Gridiron_pendulum