The ice cores appears somewhat reliable for certain questions, e.g. how high were the CO2 values in the winter on the poles 1000 years ago.

The link does not at all answer the main question for beginning to evaluate a temperature reconstruction of 125.000 years, that is: how well are the global annual temperature estimated from ice cores, for the years where we do have instrumental data?

In that link, it says ice cores track CO2. Temperatures were compiled from other geological records.

Ice cores are one of the data sources used in temperature reconstruction. From the linked page:

By measuring the ratios of different water isotopes in polar ice cores, we can determine how temperature in Antarctica and Greenland has changed in the past.