(Sorry. I take it you would like them to be complete garbage.)
There are two main ways in which a model can give bad results.
1. It can be a fundamentally bad model. E.g., Kelvin had a perfectly reasonable model of the long-term evolution of the temperature of the earth from which he deduced that it couldn't be more than a few hundred million years old, probably less. The problem was that he didn't know about radioactivity, which turns out to make a tremendous difference.
2. It can be a reasonable model but the parameters can be hard to estimate accurately, and this can lead to inaccurate predictions. In some cases, small errors in the parameters lead to large errors in the predictions. For instance, if you try to forecast the weather a year ahead then you will get all the details wrong.
This article is about the second kind of problem, and that's the one that shows the most dramatic degradation as you look at longer and longer intervals.
Different properties of the model output can show very different sensitivities to the inputs. For instance, suppose you make a model of the solar system and try to predict where the planets will be ten million years from now. I expect (though I haven't actually done the relevant calculations) you will find that tiny errors in your measurements of the planets' present positions and momenta will lead to very large errors in estimating how far around their orbits the planets are, and probably also in estimating the precession of the orbits. But I bet that almost every simulation run will put the planets' distances from the sun at about the same values, which will be about the same as they are now, and I would bet very heavily that those distances will be a pretty good match for reality. (Barring utterly out-of-model events like the human race getting technologically advanced enough to move planets around and finding it useful to do so.)
Or suppose you make a model of the weather and try to predict things five years from now. The model will do very poorly at predicting which days will be rainy in (say) New York, and if you run it many times with tiny changes in its parameters it will give completely different predictions. But it will very consistently tell you that in New York it'll be warmer in July than in January, and it will almost certainly be right; it will do pretty well at predicting rainfall in the Sahara Desert, and it will almost certainly be right about that. Some things behave chaotically, some things not so much.
So, what about those climate models? Weather is chaotic: small variations quickly lead to huge changes. Climate is much less so: to whatever extent the models are fundamentally accurate, small changes to the parameters tend to produce small changes to outputs like global mean temperature ten years later.
How fundamentally accurate are they? They seem to be pretty decent; when you go back and look at the predictions of old models for what are now past years, they do quite well. (To predict the climate you need to predict human behaviour, because human behaviour affects the climate. Some of the predictions of human behaviour that went into the models were wrong. So you can get a sort of halfway house by running an old model but using now-historical information about things like how much CO2 we would put into the atmosphere. Unsurprisingly, this improves their performance relative to just using the old models as they stand.) See e.g. https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019... for some analysis.
So: no, the sort of effects described in the article don't mean that climate models are complete garbage. The other sort of problem (fundamental errors in the model itself as opposed to its parameters) could make them garbage, but the evidence available so far suggests that even decades-old models were pretty good in this respect.
The right answer is that, just like COVID models, of course the climate models will turn out to be completely wrong in the exact details, in the scale, in the timeframe, and possibly even in the endpoints.
COVID is actually supremely easier to predict than climate. The two fundamental problems with most COVID models are that 1) the positive test counts don’t mean what you think they do, and 2) because of that, everything that is based on positive test counts being relatively meaningful is therefore totally wrong.
The climate models, since they were created by humans and not God, will likewise have almost innumerable errors which will cause all sorts of problems both minor and major with their predictions. This is universally true with all models of complex systems, and therefore should be totally uncontroversial, unfortunately the models have become like sacred cows.
Personally, just like I was with COVID, I am confident that paradigm-shifting events will intercede that will totally invalidate the model one way or another.
For COVID the paradigm shift was that the majority of cases are asymptomatic. Most people are still coming to terms with this. I heard it recently best described to think of SARS-CoV-2 as actually two viruses. One, much less prevalent, is serious and deadly. The other, much more prevalent, is mild to the point of being unremarkable. Getting one confers immunity to both. This is, I stress, a mental model, not a scientific explanation of what’s actually happening (which does not presently exist).
The point is that the things that make models wrong are often “external” to the model, the unknowables. For Kelvin it was radioactivity. His model was total “sound” just like the climate models are “sound” in terms of everything that is presently knowable. But complex systems, particularly ones with humans in the loop, have a way of surfacing curveballs the model did not, and could not, anticipate which have outsized effects in the endpoints.
The lesson with climate, just as it is being shown with COVID, is that dramatic action to address the problem (“flatten the curve”) with almost incomprehensible direct costs does not pay off.
With climate, just like with COVID, a less blunt instrument will achieve a better outcome for orders of magnitude less cost.
The sad part is that the top 2% couldn’t give a fuck about the cost either way. It’s the bottom 75% which will suffer from terrible policy.
Surely the lesson from Covid-19 is "early intervention works". Compare for eg the number of deaths in Japan/Australia/South Korea with Italy/US/UK.
Humans have a tendency to see the thing in front of them and claim, I did X, and then Y happened, so X must have helped cause Y. When in fact no such thing has been proven.
I think there are a lot of things different about those examples you gave, and absolutely no scientific way to draw a conclusion about a particular intervention having a particular effect.
Here’s an interesting comparison between Australia and Canada for example, who both responded similarly but have ended up in markedly different situations. [1]
So instead I would offer a different framework for the discussion.
In the end there are only two simple questions with COVID. 1) When we hit the endpoint, what is the total prevalence in the population, and is it lower than the natural saturation point?
So my guess for #1 is that at the endpoint, we will see that basically everyone who could have gotten it did get it. To put it another way, I suspect only the highest risk population taking the most extraordinary precautions will end up not having antibodies to SARS-CoV-2, everyone else at some point being naturally exposed.
In other words, I assume we will reach whatever the top of the S curve for COVID would naturally be based on the biological nature of the virus. Essentially, I am assuming that vaccines will not play a large role, because it will have played out before they become available, and/or because we don’t have any coronavirus vaccines and it’s probably harder to make one than we think.
2) How long do we stretch things out to get to that endpoint, and therefore, how many lives are saved through increased availability of medical care and, on the flip side, how much economic cost is incurred?
My hope here—based on some very preliminary data—is that we are actually much further along than the testing might indicate (e.g. 5% overall prevalence and much higher in certain metros, vs 0.2%)
From a health care perspective I think that the only effective treatments we have today are minimally invasive and extremely scalable — oxygen therapy and frequent repositioning. We are learning that ventilators have proven to be basically lethal to COVID patients, therefore increased availability and usage of ventilators presumably just increases the death rate. There have been no controlled studies either way at this point so no one can say for sure.
At this point because we have no effective cure or treatment, and because initial antibody testing seems to indicate COVID is extremely widespread despite the shutdown, I am very skeptical of the current approach paying off both in terms of delaying spread & saving lives through higher availability of care.
[1] - https://www.theglobeandmail.com/opinion/article-canada-and-a...
Well no - Australia required quarantine of all returning overseas travellers from March 15, Canada advised self isolation from March 25. That's a huge difference.
Again - correct, thoughtful interventions worked.
> Essentially, I am assuming that vaccines will not play a large role, because it will have played out before they become available, and/or because we don’t have any coronavirus vaccines and it’s probably harder to make one than we think.
I've seen the "we don’t have any coronavirus vaccines" meme too. People don't seem to realise we've only had one serious coronavirus before (SARS) and it isn't anywhere as nearly contagious. There just hasn't been much work done on coronavirus vaccines before.
We do however have anti-viral treatments (eg Tamilflu), and it's likely we'll either get a working vaccine or a anti-viral drug that provides some relief in the next 12-18 months.
> My hope here—based on some very preliminary data—is that we are actually much further along than the testing might indicate (e.g. 5% overall prevalence and much higher in certain metros, vs 0.2%)
Either way, it's a long way off the required 80% for herd immunity.
> We are learning that ventilators have proven to be basically lethal to COVID patients
This is absolutely untrue. There have been people sharing data showing that the mortality rate of people on ventilators is higher than those who aren't, but that is because ventilators are only used on people who are closer to death.
It's true that in some cases oxygen is sufficient for some people.
The point is a careful and thoughtful examination shows that a country’s geographic, social, and economic “natural affordances” are largely responsible for the differing COVID case loads, much less than the effective dollar amount “spent” on interventions. Spending on an intervention includes the economic hit as well as the direct cost. E.g. Australia was able to get a lot more bang for their buck because they’re an island and didn’t have a large segment of their population traveling in hotspots.
I think we agree on the vaccines. The point isn’t there’s something specific about coronavirus. It’s not a “meme” that we don’t have a coronavirus vaccine. We actually don’t have one. That just means we might not have as much experience ramping up a new one as we do, for example, with influenza, and there may be things we need to learn along the way.
A working vaccine or treatment in 12-18 months is a long way off.
If we are anywhere near 5% prevalence that means we are a lot closer to 65% (when herd immunity starts being noticing beneficial) than you might think, since it is non-linear.
Regarding ventilators, even in the general (non-COVID) case if you find yourself on one you’re about 50/50 not going to walk away from it. But with COVID it appears ventilators exacerbate the swelling in the lungs and in NY 80%+ of ventilated patients have died. [1]. That’s what I mean by “proven to be basically lethal”. We know the vast majority of COVID patients on ventilators die, far more than just the average ventilator death rate for pneumonia, using the same O2 saturation markers.
So doctors are moving away from ventilators and no longer using the typical O2 saturation cutoffs for ventilating because interestingly patients with lower O2 levels are not in the typical level of distress. It seems the CO2 is still getting out even though the oxygen is not getting in... more like hypoxia.
> It's true that in some cases oxygen is sufficient for some people.
We can agree on more than that. Luckily the fact is that oxygen therapy is sufficient for the vast majority of cases. Just do the math on hospitalizations and deaths in NY with the known variable that 80% that were ventilated died. QED the vast majority were not ventilated.
[1] - https://www.nytimes.com/aponline/2020/04/08/health/ap-us-med...
I was referring to the idea that is going around that "we don’t have any coronavirus vaccines" is somehow significant and means it is especially hard to make one.
> Regarding ventilators
I think the NY Times article is good, but it's important to note that this is evolving and treatments are changing. But saying "some doctors say they're trying to keep patients off ventilators as long as possible, and turning to other techniques instead" is very different to "ventilators have proven to be basically lethal to COVID patients".
That makes ventilation an absolutely last ditch effort. It means if the ~60,000 ventilators already in US hospitals before this started were each used just once ever to ventilate a COVID patient, you would expect 48,000 of those patients to die.
I’m sure this also has a pretty significant effect on the ICU utilization rate as a percentage of hospitalizations.
And this is besides the fact that early serological testing points to the overall hospitalization rate being significantly lower than 10%.
You seem to believe that the ventilators are causing deaths. There's plenty of evidence to show that once Covid-19 takes hold it is particularly viscous because of the number of things that it does to the body.
Some of the things that go wrong in some cases include: lung damage (obviously), kidney damage, liver damage, strokes, encephalitis, seizures, heart damage, meningitis, bloody diarrhea etc[1].
I agree the death date of people on ventilators are horrifying. But I don't think there is much evidence that ventilators are causing that rather than being administered to people who already have serious symptoms.
[1] https://www.sciencemag.org/news/2020/04/how-does-coronavirus...
All we have is anecdotal reports, plus the fact that significantly more COVID patients die on ventilation than the average pneumonia patient who is [invasively] ventilated.
Since doctors were ventilating patients at the same O2 saturation points as usual, we can say at least that something is markedly different with COVID induced ARDS from typical ARDS that makes invasive ventilation particularly insufficient, if not actively harmful.