> What COVID forecasters can learn from climate models

That they don't work?

I have been working with simulation and models for the past 25 years, and models are just that models. They provide a picture of what you want to see, not the truth.

Do you have an alternative to suggest?

He's not saying that models should not be used but that they should be used thoughtfully. Brainstorming in groups helps at first, but later sessions where you play Devil's Advocate to critique models and/or re-evaluate are also useful.

The headline ignores the fact that the covid forecasters have changed their model over time to be basically what the paper said to (based on the initial model from March done in great panic). So this story is really nothing. Models improve over time if you have enough time.

Climate models? They are so extremely bad, scientifically.

I'll guess not much - since the climate models have been consistently wrong. But since Covid was politically weaponized in 2020 just as climate science was politically weaponized in the 1990s, I suppose we can learn something about Covid forecasters from their climate counterparts.

> politically weaponized

If a political leader's response to a quarter million deaths is denial, that's not weaponization, that's democracy working.

> because researchers didn’t appreciate how sensitive CovidSim was to small changes in its inputs, their results overestimated

As a software engineer who has written simulations before, I can assert that this is almost always the case with simulations. The more complex the phenomena is that you're trying to model, the more likely it is that the simulation is completely wrong because you got a single parameter slightly off.

That's why I'm skeptical of climate simulations too because they are insanely complex and they're certainly missing a lot of parameters. For example, just off the top of my head, I wonder if climate models factor in the fact that tree populations will evolve to adapt to increased CO2... I don't think they can model evolutions of tree populations because it gets into genetics. Which is a level of complexity far beyond what even the top supercomputers can simulate and even then, a lot of things would have to be simplified to the point that the model ends up having nothing to do with the real world. The butterfly effect is one of the reasons why complex simulations are not useful in predicting anything.

Simulations of complex phenomena are completely useless IMO.

You can always cherry pick one simulation out of thousands which is reliable for a few years in a row but they fall apart sooner and later so you end up using a different simulation every few years... And people keep coming up with excuses every time saying that the supercomputers were not fast enough the year before.

> the fact that tree populations will evolve to adapt to increased CO2

Many climate models only aim to predict the next century or two because that's what's relevant for humans. Trees have long generation cycles and evolution tends to take many generations to show significant effects. So on geological timescales they will probably adapt, but considering the speed at which AGW is happening that adaption will be too late to be of interest.

For climate the simulations can at least back tested against the past climate.

It's unclear how that can be done for for Covid. There's nothing to test against because both the virus and the society's reaction is very different.

But the back testing is useless because you can just have a lot of models and cherry pick all those models which happen to match historical data. Then each year some of those models are proven wrong and the attention shifts towards models which happened to get it right... And new models are invented constantly. If you have enough models and companies working on them, you will always find a few which just happen to have guessed all the numbers correctly. It's like lottery. Out of millions of people, some people will get all the numbers right.

Yes, back testing is not perfect, but it's better than nothing.

In the Covid simulation case they have nothing as far as I can tell.