He meant one that is physically based and interpretable, rather than a black box based on pattern matching.

But I'll take the accurate black box any day, at least for weather forecasting. Climate modelling is a totally different thing.

Accurate black box is fine until it blows a forecast and people die, and you want to fix it for the next time.

Take the situation with Hurricane Otis — what do you do if an AI doesn’t detect cyclogenesis 24-48 hours beforehand? Are we sure tuning the model to detect this event will improve forecast skill in general, or will it make it worse?

Weather forecasting models will gradually improve over time but it's unreasonable to expect perfect accuracy. That is mathematically impossible in a chaotic system. Some people will always die in extreme weather events.

Doesn't exactly answer your question, but maybe this link will give enough intuition so that i can be lazy and hand-wave away an answer

https://en.wikipedia.org/wiki/The_Book_of_Why#Chapter_1:_The...