How many people can scrutinize the esoteric ensemble of climate models that existing climatologists use to generate weather predictions today?

The climatologists can, at least. Can they scrutinize the esoteric ensemble of weights making up this AI model? And which type of model's going to be easier to update based on changing climate parameters, a climate model or an AI model?

Weather and climate models have their own physics, which at the very least means that the solution is physical for the universe that particular model inhabits. The boundary conditions are parameterized, and those can be tweaked as climate and land use changes.

AI models don’t have any of that, but they are actually more akin to human forecasters, gaining forecast skill from pattern recognition. I think there’s a place for both in weather forecasting, but I’d have zero confidence in an AI climate model, or anything longer than a year. An AI might be very good at seasonal forecasts though, picking up easy to miss signals in the MJO or ENSO.

> Weather and climate models have their own physics, which at the very least means that the solution is physical for the universe that particular model inhabits. The boundary conditions are parameterized, and those can be tweaked as climate and land use changes.

That really isn't true these days. The dynamical cores and physics packages in numerical weather prediction models and general circulation models have more-or-less converged over the past two decades. For instance, you'll find double-moment microphysical schemes in a cross-section of both classes of models, and slightly specialized versions of full-fledged GCMs can be be run within assimilation frameworks to generate true-blooded weather forecasts.

> AI models don’t have any of that, but they are actually more akin to human forecasters, gaining forecast skill from pattern recognition

This grossly sells short what the current crop of AI weather models is capable of, and how they're formulated. It's best to think of them as "emulators" of their physics-based cousins; they're trained to reproduce the state transitions from t=t0 to t=t0+delta_t that an NWP system would generate. It's a bit reductive to call this "pattern matching", especially when we increasingly see that the emulators recover a fair bit of fundamental dynamics (e.g. Greg Hakim's work which reproduces idealized dycore tests on AI-NWP models and clearly demonstrates that they get some things surprisingly correct - even though the setups in these experiments is _far_ from real-world conditions).

> That really isn't true these days. The dynamical cores and physics packages in numerical weather prediction models and general circulation models have more-or-less converged over the past two decades.

Ah, well, I did stop studying GCMs about 20 years ago so perhaps I should shut up and let other people post. I appreciate the detail in your explanation here, and I wouldn’t mind a link to papers explaining the current state of the art.

I'm not sure I can point you to a single reference, but a good starting point would be the UK Met Office's "Unified Model", which provides a framework for configuring model simulations that scale from sub-mesoscale rapid refresh (e.g. the UKV system) to traditional global modeling (e.g. MOGREPS) and beyond into climate (latest versions of the Hadley Centre models, which I think the current production version is HadGEM3).

More than 0

Climatologists use climate models to predict the climate. Meteorologists use weather models to predict the weather. They are different time scales and disciplines.

I think it certainly doesn’t foreclose on knowing more in the future, based on what we observe with this new model.

But that's the point of a weather forecast. We don't do them for the science. (I'm not arguing against weather research.)

I would think we also use these models to run simulations. So maybe the AI models can be used to run simulations with different kinds of inputs to see if doing X in one place (like planting a lot of trees in one area) will have an outsized impact rather than doing Y in another place.

AI models rarely have that kind of predictive power because they are noticing patterns not running simulations.

The model may care about trees because mountains above a specific height don’t have trees on them. The old, measures stop being useful once you start optimizing for them, AI edition.

Is that similar to being uncomfortable with quantum mechanics and the shut-up-and-calculate advice?

whelp...

Ok, how would you articulate an understanding of how the weather works?

Let's face it, it's not going to be easy to frame in natural language...

You can take a meteorology course that explains how the weather works. That course is the articulation.

That’s great but it’s not going to scale to the level of detail that makes this better than traditional meterologist models

do you want a causal weather model or one that can predict the weather well?

When climate change occurs (which it is), we're going to want a causal one so we can actually make a forecast instead of predicting only based on past (unchanged) data.

What does a causal weather model even mean?

He meant one that is physically based and interpretable, rather than a black box based on pattern matching.

But I'll take the accurate black box any day, at least for weather forecasting. Climate modelling is a totally different thing.

Accurate black box is fine until it blows a forecast and people die, and you want to fix it for the next time.

Take the situation with Hurricane Otis — what do you do if an AI doesn’t detect cyclogenesis 24-48 hours beforehand? Are we sure tuning the model to detect this event will improve forecast skill in general, or will it make it worse?

Weather forecasting models will gradually improve over time but it's unreasonable to expect perfect accuracy. That is mathematically impossible in a chaotic system. Some people will always die in extreme weather events.

Doesn't exactly answer your question, but maybe this link will give enough intuition so that i can be lazy and hand-wave away an answer

https://en.wikipedia.org/wiki/The_Book_of_Why#Chapter_1:_The...

ahhhh i was hoping someone wouldn't mention this. i wanted to add a caveat but it made my comment look ugly lol.

but like someone else says weather and climate models forecast on different scales and for different purposes usually.

Nothing prevents people from training new models.

And note that these new models based on machine learning are already better at predicting extreme events. This is because existing models are not built entirely from first principles but rather include a lot of heuristics to describe important phenomena like cloud formation and albedo effects. That means that traditional models are just as rooted in weather-as-it-was as the machine learning models are. The big difference is that it takes a lot of work to identify the dependencies in traditional models while it takes less work to retrain the machine learning model.

The current crop of AI weather models captures a surprisingly robust array of "core dynamics". E.g., recent work by Hakim and Masanam [1] reproduce classic dynamical core test cases on AI weather models and show that in many cases they robustly capture the expected dynamical responses to these highly idealized setups - setups far outside the bounds of what the models are trained on. What this means is that the causal or dynamical mechanisms producing a particular flow regime can very directly be inferred from the model.

[1]: https://journals.ametsoc.org/view/journals/aies/3/3/AIES-D-2...

thanks for providing evidence that these two things don't need to contradict.

I think this will be the next generation of science to some extent. The things we can understand and call explanations/reasons might be something involving 5 or 50 variables with not too many interactions between them. They have some unifying principles or equations we can fit in our head. I think many things inherently just involve far too many variables and complexity for us to understand in a neat theory and we are hitting those limits in biology & physics. Even so I'm sure we will develop better and better ways to interpret these models and get some level of understanding. Maybe we can understand them but not create them past a certain scale.

This description isn’t how science has played out. When the scale changes often the model gets simple again.

Think about how complex friction is at a molecular level. But a single coefficient is a good enough model for engineers and a continuous 1d graph is incredible.

There is also no evidence that general AI models like multilayer perceptrons are good at constructing physical models from phenomena and lots of examples where they aren’t.

The opposite seems to have had more success. Someone who understands a system constructs a model and then lets a computer determine the actual parameter values.

We can work backwards from correct predictions and develop an understanding that way.

But most people just need to know if it’s going to be storming, hot or, going to rain on a given day and that is where this shines.

We can always work backwards, regardless of AI.

Sure, but with this new predictive model we will have better predictions to work backwards from.

OC was saying (I’m going to paraphrase) that this is the death of understanding in meteorology, but it’s not because we can always work backwards from accurate predictions.

Or we could wait 15 days and work backwards from what the weather actually turned out to be.

I guess there could be some value in analyzing what inputs have the most and least influence on the AI predictions.

Comparing the difference between correct predictions and incorrect predictions, especially with a high accuracy predictive model, could give insight into both how statistical models work and how weather works.

Define "our understanding". With complex / chaotic systems there sometimes are no higher level laws that govern them - all we have is just modeling and prediction.

A better name for "laws" is "models."

A fisherman was relaxing on a sunny beach, enjoying the day with his fishing line in the water. A businessman, stressed from work, walked by and criticized him for not working harder.

"If you worked more, you'd catch more fish," the businessman said.

"And what would my reward be?" asked the fisherman with a smile.

"You could earn money, buy bigger nets, and catch even more fish!" the businessman replied.

"And then what?" the fisherman asked again.

"Then, you could buy a boat and catch even larger hauls!" said the businessman.

"And after that?"

"You could buy more boats, hire a crew, and eventually own a fleet, freeing you to relax forever!"

The fisherman, still smiling, replied, "But isn't that what I'm already doing?"

But AI models in no way free you from work and they are hardly relaxing.

If anything, they "free" people from understanding but that's an activity that many if not most people value highly.

I think that might have been their point. People moving work to AI so they can "relax" by working on more complicated technical matters (or more AI) are the businessmen, and the meteorologists just chilling out predicting the weather as best as they can with science are the fishermen.

Edit: Just saw their reply to you, so maybe I was wrong about the parable coming across wrong.

Okay.

I mean, yeah, if you want 90% of your relaxation time for the rest of your life to be while you're fishing, that's fine.

For the businessman to assume otherwise is not outlandish. The idea of an entire fleet is overblown, but having "fuck you money" is a pretty nice goal.

-

Also I don't see how this applies to meteorologists? Which part is the "working more" aspect?

This is a true story and why they have never built a bridge between Sicily and Italy even after thousands of years.

This has been the case for years now, way before the AI craze. We just used to call it machine leaning. The best performing predictive models are black boxes which can’t practically be interpreted by humans the same way you can say a linear regression model that gives easily digestible parameters as output. Boosted trees are a great example of very well performing models that quickly become impossible for humans to understand once they get big enough to be useful.

What is there that we don't already understand? Sure there are probably higher level patterns in the weather that the model might be exploiting to make the claimed better predictions, but normal models are just based on physics and people rarely cared to try and improve them based on these patterns before. Mainly because they are too complex.

In short those patterns are only useful because of AI.

something interesting is that these predictions can be made, and gives the hypothesis "we can have a better understanding of weather than makes better predictions than our current understanding does"

in contrast to "we cannot make better predictions on this input data than what we're doing now"

You will be mocked by the AI hype crazies for your very serious and important question during this time. The reality is no ,we won't learn why and how something works. And it will get much worse in the future. We are trading human learning for machine learning. As time goes on humans will get more stupid and machines more intelligent and eventually we will have generations who know nothing about how anything works and depend on the machines to tell them how to function.

One could make the argument: hey... fifty years ago everyone knew intricately how a car worked because you had to. It broke down so often, you needed to be able to repair it yourself on the side of the road. Now people just press a button and if it doesn't work you have the 'shop' take care of it for you. AI advancement will be no different. Problem is: the 'shop' today is still humans who designed and built the cars and know how a car works and how to repair one. AI advancement can lead to eventually no one knowing anything as models get so sophisticated we just don't know why A leads to Z.

The problem itself is a bunch of random weights. There is no clean structure to discover that explains it.

I’m by no means an expert in weather forecasting, but I have some familiarity with the methods. My understanding is that non-“AI” weather models basically subdivide the atmosphere into a 3d grid of cells that are on the order of hundreds to thousands of meters in each dimension, treat each cell as atomic/homogeneous at a given point in time, and then advance the relevant differential equations deterministically to forecast changes across the grid over time. This approach, again based on my limited understanding, is primarily held back by the sparse resolution and the computational resources needed to improve it, not by limitations of our understanding of the underlying physics. (Relatedly, I believe these models can be very sensitive to small changes in the initial conditions.) It’s not hard to imagine a neural net learning a more efficient way to encode and forecast the underlying physical patterns.

FWIW, the UK Met Office's models are described here: https://www.metoffice.gov.uk/research/approach/modelling-sys...

> It’s not hard to imagine a neural net learning a more efficient way to encode and forecast the underlying physical patterns.

And that is where your understanding breaks down.

What makes weather prediction difficult is the same thing that make fluid-dynamics difficult: the non-linearity of the equations involved.

With experience and understanding of the problem at hand, you can make some pretty good non-linear predictions on the response of your system. Until you cannot. And the beauty of the non-linear response is that your botched prediction will be way, way off.

It's the same for AI. It will see some nicely hidden pattern based on the data it is fed, and will generate some prediction based on it. Until it hits one of those critical moments when there is no substitute to solving the actual equations, and it will produce absolute rubbish.

And that problem will only get compounded by the increasing turbulence level in the atmosphere due to global warming, which is breaking down the long-term, fairly stable, seasonal trends.

This is true for many but not all weather models.

GFS and IFS are both medium-range global models in the class Google is targeting. These models are spectral models, meaning they pivot the input spatial grid into the frequency domain, carry out weather computations in the frequency domain, and pivot back to provide output grids.

The intuition here is that, at global scale over many days, the primary dynamics are waves doing what waves do. Representing state in terms of waves reduces the accumulation of numerical errors. On the other hand, this only works on spheroids and it comes at the expense of greatly complicating local interactions, so the use of spectral methods for NWP is far from universal.

In Australia, meteorologists used to be deployed across the country to local offices and would receive computer-generated forecast models (and other raw data) whenever the supercomputer at headquarters had finished running a job. The local meteorologists would then be allowed to apply their local knowledge to adjust the computer-generated forecast.

This was (and still is) particularly important in situations such as:

* Fast moving weather systems of high volatility, such as fire weather systems coupled with severe thunderstorms.

* Rare meteorological conditions where a global model trained on historical data may not have enough observed data points to consider rare conditions with the necessary weighting.

* Accuracy of forecasts for "microclimates" such as alpine resorts at the top of a ultra-prominent peak. Global models tend to smooth over such as an anomaly in the landscape as if the landscape anomaly was never present.[1]

It'd perhaps be possible to build more local monitoring stations to collect training data and run many local climate models across a landscape and run more climate models of specific rare weather systems. But it is also possibly cheaper and adequate (or more accurate) to just hire a meteorologist with local knowledge instead?

[1] Zanchi, M., Zapperi, S. & La Porta, C.A.M. Harnessing deep learning to forecast local microclimate using global climate data. Sci Rep 13, 21062 (2023). https://doi.org/10.1038/s41598-023-48028-1 https://www.nature.com/articles/s41598-023-48028-1

This whole article smells like marketing and a way to monetize their weather model. I hope its leaps and bounds beyond current capability but I have very strong doubts about its efficacy in the real world.

Looks like it will be free and open

> We’ll be releasing our model’s code, weights, and forecasts, to support the wider weather forecasting community.

https://deepmind.google/discover/blog/gencast-predicts-weath...

> "We'll soon be releasing real-time and historical forecasts from GenCast, and previous models, which will enable anyone to integrate these weather inputs into their own models and research workflows."

They will give you the weights and code not the forecast - your quote is incomplete.

Its either a temporary gift to the community until its adopted then charge for it OR they know most orgs can integrate that into their products therefore requiring to buy google products IF it works as they say it does.

With AI we risk treating the system like a black box

I'm not sure there's a good WHY. To me both weather and AI are similarly complex and inscrutable, so they seem right for each other.

We could train AI models to simplify models in ways that require much less "intelligence" in the given domain.

For instance, we could ask AI to simplify the "essence" of the problems it solves in a similar manner to how Einstein and Feynmann simplified laws of Physics. With train/elevator metaphors or representations like Feynmann diagrams.

Of course, such explanations don't give the depth of understanding required to actually do the tensor calculus needed to USE theories like General Relativity or Quantum Electrodynamics.

But it's enough to give us a fuzzy feeling that we understand at least some of it.

The REAL understanding, ie at a level where we can in principle repeat the predictions, may require intuitions so complex that our human brain wouldn't be able to wrap itself around it.