and if the earth speeds up or slows down we loose sync with far off time keepers who dont know yet ?

mostly this is tektonic re-adjustments on a minor level or asteroid strike on a major level. of course you could say that an asteroid strike would be a larger problem than re-syncing the stone clock