Models are supposed to be based on first-principles reasoning. A "model" that's not designed to capture some aspect of reality is called a computer game...

The Imperial model gives reproducible results, so most of the noise was professional programmers being salty about the fact that ugly code can still be accurate code.

The fact that it predicted things like 80% of the population becoming infected was simply unrelated to code quality.

Regardless of code quality, are these models really useful if, as you say, they are "accurate", but still make predictions wildly out of step with reality?

Usually, because they help narrow down the sources of error. If you've tested your code so you're sure the various bits work as intended, any remaining somewhat crazy results are down to things you don't understand properly.

A somewhat validated code allows you to do by computational experiments what you can't by real life experiments. Very handy if one of the common end states in the experiments is "death".

The real imperial story is that there were two wrong parties: - The scientists who were not including enough detail in their math. - The drive-by developers who had no idea what did or didn't make scientific simulations work.

Together they produced both an inaccurate estimate, and a github page full of uninformed speculation about what the problem might have been. ;)

Honestly I think the main problem was they choose overly pessimistic estimates, or maybe that their model is so expensive they could only investigate a few choices of the parameters.

same way that a given implementation is better than none. Its a concrete implementation that serves as an anchor. Its a fixed point to discuss implementation issues around as well as working as a harness that others can build on/around.

Wasn't it more professional programmers being salty that policy decisions were being made by code with absolutely no tests or peer review?

Code with no _unit tests_ or _code reviews_ possibly, it was obviously tested by the scientists that it made somewhat sensible and self-consistent predictions.

In addition, predictions by earlier versions of the code had been peer reviewed, which is about as good as it gets outside of computer science.

The report itself wasn't formally peer reviewed either, though in the crisis mentality of March this year not many people had the patience for peer review. Frankly, most still don't if you consider the amount of news about unreviewed science that get published.

gives? perhaps it does now, but it was certainly non-determinstic when it was first released, which is also when it was used for policy decisions.

See for example https://news.ycombinator.com/item?id=23099386

But how useful was the model? It showed that the virus would spread exponentially, but we knew that. Is there anything that it showed that we didn't know when they programmed it, with constants tuned to what they expected?

That was the point. They used stochastic partial differential equations, they're by definition non-deterministic.

No, they should be reproducible if run with the same seed.

Reproducibility doesn't mean accurate or good, it means it can be recreated. Its a crucial first step, but a model can be reproducible and also totally worthless.

Yeah, but as you can see in this thread the rock star programmers of HN only know enough epidemiology to complain about code formatting and improper RNG seeding.

An interesting article. I was unaware that John Carmack worked on the code before it was released.