You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks
This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies.
If you are unlucky- your use case may be one where Evals require human review.
Unless you are YOLOing it without evals. In that case this is relevant.