I'm pretty certain that scientists are able to reconstruct temperatures going back thousands of years from various sources, with other information such as atmospheric carbon volumes. Here's one source: https://www.bas.ac.uk/data/our-data/publication/ice-cores-an...