http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf (which was pointed to downthread) indicates tokenizing and converting to scientific notation gives a 5% increase in performance on a "comparing numbers" benchmark.