flwi
flwi on Hacker News
Define "fundamental", but I added skills to run complicated evaluation procedures for my ML research. This way I can open 5 CC instances, let them run and iterate on research without intervention. After that, I can do the final review.
Hey author here. Your argument is completely valid, we only model physics implicitly and thus have no prove that the model "actually knows the physics". Practically, this might not matter much: If the model can predict the evolution of the system to a certain accuracy, the user won't care about the underlying knowledge. And even for modern physics (quantum / GR), we know we miss something and yet, the models we have are incredibly useful.
On a tangent, we cannot prove that LLMs actually know language, yet they can be incredibly useful.
Of course, a true world model would be much nicer to have, I agree with that!
I haven't seen electromagnetic systems included yet, probably since they are less training data for it.
In your chase, with such specific systems, a model trained only on your data might make more sense, though
The Polymathic AI company does a lot of stuff in that direction.
Can you point me to the papers? In general, faster dynamics and chaotic systems are probably the hardest. Of course combined with long-term stability
Very interesting paper! We did not run this model ourselves.
From what I've understood, the results are in the same order of magnitude, but the model is 4x the size. And (similar to all other predecessors), they finetune on new physics instead of zero-shot
Author here: we do NOT do conservation of energy/momentum. We are currently trying to incorporate that in a follow up paper, but for now, the models that try that (e.g. PINNs (soft constraint) or hard constraint models, all perform badly when modeling multiple systems.
Perhaps, we will encounter the bitter lesson again and a well trained model will solve this. But as I said, we are also looking at hybrid models
Author here: we do NOT do conservation of energy/momentum. We are currently trying to incorporate that in a follow up paper, but for now, the models that try that (e.g. PINNs (soft constraint) or hard constraint models, all perform bad when modeling multiple systems.
Perhaps, we will encounter the bitter lesson again and a well trained model will solve this. But as I said, we are also looking at hybrid models
Author here,
Wow, I didn't think this would HN. I actually planned to do the advertisement rounds only after the final ICLR submission.
This is our attempt at creating a model which understands multiple physics, which is in contrast to PINNs and Neural Operators, which focus on much more narrow systems.
Obviously, the biggest issue is still data (3D and real-world problems), but I think we and a few other groups make significant progress here.