PontifexCipher
- Build a complex intractable mathematical model (here, Navier-Stokes)
- Approximate it with a function approximator (here, a Physics Informed Neural Network)
- Use the some property of function approximator to search for more solutions to the original model (here, using Gauss-Newton)
In a sense, this is actually just the process of model-based science anyway: use a model for the physical world and exploit the mathematics of the model for real-world effects.
This is very very good work, but this heritage goes back to polynomial approximation even from Taylor series, and has been the foundation of engineering for literal centuries. Throughout history, the approximator keeps getting better and better and hungrier and hungrier for data (Taylor series, Chebyshev + other orthogonal bases for polynomials, neural networks, RNNs, LSTMs, PINNs, <the future>).
You didn't say anything to the contrary, and neither did the original video, but it's very different than what some other people are talking about in this thread ("run an LLM in a loop to do science the way a person does it"). Maybe I'm just ranting at the overloading of the term AI to mean "anything on a GPU".
- v2/v3 (not r1) seem to be cloned from o1/4o output, and perform worse (this cost the oft-repeated 5ish mm USD)
- r1 is specifically a reasoning step (using RL) _on top of_ v2/v3 and performs similarly to o1 (the cost of this is _not reported anywhere_)
- In the o1 blog post, they specifically say they use RL to add reasoning to LLMs: https://openai.com/index/learning-to-reason-with-llms/