I know it's not a simple answer, but how would you embed checks to flag or highlight potentially incorrect results to the user?

For physical processes, you can sometimes lean on conservation laws. Financial, no-arbitrage "laws". Sometimes it can help to run two different models, or different numerical methods, and compare their results. More generally, it's very, very hard. I am fairly sure there are multiple published results in computational fluid dynamics that are subtly wrong.

A lot of computer models are deliberately wrong but useful as well which is another ball game in terms of analysing results.

> I am fairly sure there are multiple published results in computational fluid dynamics that are subtly wrong.

I wonder what that means for the accuraccy of the climate models...

All models are wrong, some are useful.

It's probably impossible to accurately simulate the climate of the planet decades into the future.

That does not meaningfully detract from the evidence for human caused global warming however.

I have to applaud the subtle elegance of the sarcasm here

No sarcasm intended. The science of climate change rests on many legs. It was already established long before massive models on the scale we have today were possible. Computer simulations attempts to understand how it will affect us, but does not contribute much evidence.

They are all wrong. Historically they've overpredicted warming, and then the models are updated to fit the (new) data better. They perform worse in areas with less data, like polar regions.

To forestall knee-jerk downvotes: I'm not saying climate change isn't real or that anthropogenic global warming doesn't exist. I'm saying the models are not yet developed enough to predict very accurately. Early heliocentric models made poor predictions too because they assumed circular rather than elliptical orbits, they were still more "right" than geocentric models.

Can you suggest good places to look for more info on this?

I have tried a few times to find info on the accuracy of these models and couldn't find much. And most models seem to be closed source.

In fact, climate models going back over fifty years have performed quite well; surprisingly so:

https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/201...

https://www.theguardian.com/environment/climate-consensus-97...

It depends on how you define "quite well". They are definitely directionally correct and in the right ballpark. This is a politically sensitive research area so the researchers are incentivized to make strong claims that you wouldn't make about, say, predicting sports outcomes or the stock market.

The first link you posted is kind of punting on the hard part by saying that the reason the models overpredict warming is because CO2 didn't rise as much as they expected so if you put the actual observed CO2 concentration in then the temperature prediction comes out closer to what was observed. But the CO2 concentration is a parameter of the model, so they didn't capture its dynamics properly and then had to retroactively change it to get the observed data.

Again, I want to reiterate that I'm not disputing the process of climate change or saying it's not a problem. I'm saying that modeling it is hard and historically the models have overestimated warming.

That doesn’t seem like a fair criticism. Most climate models are physics models. They can’t know how much CO2 people are going to add to the atmosphere. But if you tell them how we altered the composition of the atmosphere, they predict the temperature change correctly. That’s what I mean by good performance.

Part of the job of the modeler is to make good decisions about the parameters and their uncertainties. If they are systematically overestimating CO2 and getting high predictions as a result then they aren't modeling properly, unless there are exogenous reasons why CO2 is lower than expected. Which might be possible for all I know (Global financial crisis maybe? China growth lower than expected?) but the dynamics of CO2 are part of the model they are using for prediction so they can't just punt on it by saying they didn't know what CO2 would be. Prediction's hard, especially about the future, as the saying goes.

All manner of tests are done in fluid-dynamical modelling. Test suites are developed along with models. Simulations are done with a range of alternative models, or individual models with different conditions are applied, and the differences between the simulations are studied in detail. Models must rely on a fairly large set of parameterizations of processes that cannot be resolved directly, and tests are done to see which parameterizations make the most sense.

Models stem from the academic environment, not the business environment. This means that the models are open-source, so everyone can see what everyone else is doing. And 'everyone' is a pretty large set.

Note that there is no commercial element to any of this, which means there is no incentive to hide problems. It's the reverse: reputation is earned by finding problems, not by hiding them.

A large part of the challenge of climate modelling is the leveraging of increasing computer power to resolve progressively smaller scales of motion, and this uncovers the need to understand those scales in isolation. Modelling is often used at this level also.

None of this is to say that climate models are perfect. They obviously are not. But the system of academic science is very good at improving models and, importantly, exposing their limits. An indication of the latter is the pairing of uncertainties with predictions: a hallmark of this scientific community.

> I know it's not a simple answer, but how would you embed checks to flag or highlight potentially incorrect results to the user?

For starters, some problems do have analytical solutions. Yo can compare a FEM model of these problems with the known analytical solution and see if it's close enough or not. One of them is the elasto-plastic plate with a hole.

You can also run unit tests at the element level.