Re:Lagrangian Neural Networks
by qchris on Thursday March 12, 2020 @01:15AM
(#22553991)
On a personal note, I consider the university class where I learned about Lagrangians for the first time to be a pivotal moment in my life. It was the first time I experienced a fundamental sense of awe and beauty about a mathematical construct, in physics or otherwise. I'd thought physics was useful and interesting before that, but seeing derivation of and application the Lagrangian formulation to mechanical systems was, and is, simply gorgeous to me. And as someone who recently began dipping their toe into neural networks, this is really very cool.
[ Parent
| on HN ]
Chris, glad you like it. I can relate to that feeling, which I also had as an undergrad. It was a big motivation for this work -- and I think the analogy can go even further. For example, there's a really cool lecun paper about NN training dynamics actually being a lagrangian system: http://yann.lecun.com/exdb/publis/pdf/lecun-88.pdf. Another thing I want to try is literally write down a learning problem where S (the action) is the cost function and then optimize it...
That does seem interesting! After skimming that paper, I think I'm going to need to sit down with it in order to really parse through things, though. Some of the operator combinations seem to be things I haven't worked with jointly before. I'd definitely be interested to see the results of using the action as the cost function, though!
I kind of went the other way. I skipped out on my undergrad classical mechanics class and did the graduate course, taught by a guy from the Landau school, out of volume 1 of Landau & Lifshitz. So in the summer after my 3rd year I spent some quality time with Newton's Principia and then most of the rest of the summer staring off into space as I hooked up everything between the two systems again.
Man, that was a great summer.
Oh god Landau & Lifshitz, that wakes memories. That was hardcore
Wow, that's really inspiring. I also love beauty. Can anyone recommend a way for someone with undergrad level math to learn about Lagrangians?
There a lot of good resources - maybe try these to start:
https://www.feynmanlectures.caltech.edu/II_19.html
http://www.damtp.cam.ac.uk/user/tong/dynamics/two.pdf
https://theoreticalminimum.com/courses/classical-mechanics/2...
An economics PhD once asked me for help with Langangians, but at the time I was quite occupied with other branches of mathematics. I wonder if it would make sense to revisit it. If I recall, the problem he was looking at was quite standard, and involved perturbations (indeed, like the corona virus) to economical models.
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @10:12AM
(#22557040)
A "perturbation" is usually short for a "small perturbation," which SARS-CoV-2 is definitely not. ;)
Maybe usually, but iirc in his case he specifically said "major" events and recourse to normal after that.
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @10:13AM
(#22557049)
What's so great about it? I had that class too and I didn't think it was so hot. Maybe I missed something.
Elegantly deriving the optimal path, which also happens to be what reality usually does? (Because many problems are "easily" transformed into energy minimization problems [principle of least action].)
Conservation laws (and/or the associated symmetries) might be more fascinating, but that all builds on Lagrangian mechanics.
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @12:04PM
(#22558209)
Why would you say that reality "minimizes action" when you could just as easily say it "solves differential equations?" They are two equivalent ways of looking at the same thing, and you can't really tell which one nature does.
"solves differential equations" doesn't tell you anything about which differential equations. "minimizes action" gives you a way to get the differential equations corresponding to pretty much every scenario.
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @02:22PM
(#22559959)
Specifying the action is equivalent to specifying the differential equations. Knowing that you start with a Lagrangian doesn't tell you more than knowing you start with some differential equations.
Re:Lagrangian Neural Networks
by Brain_Thief on Thursday March 12, 2020 @01:45PM
(#22559559)
But are the statements "reality minimizes action" and "reality solves differential equations" actually, literally equivalent to each other? It strikes me that this is not in fact the case. There are solutions to systems describing skyscraper sway that have the building stand at a terrifying lean angle; what is supposed to be selecting the particular solutions that we observe? A purely random walk through the solution space seems unsatisfying as an answer.
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @02:26PM
(#22560003)
They are actually, literally equivalent to each other. Differential equations aren't a "purely random walk," they are a statement of what is about to happen based on what is happening. You have three formalisms: Newtonian (differential equations), Lagrangian (action minimization), and Hamiltonian (differential equations, but derived from a scalar field kind of like in Lagrangian mechanics). They are all different ways of writing down the same thing, and their advantages and disadvantages are related to situational convenience.
Re:Lagrangian Neural Networks
by Brain_Thief on Thursday March 12, 2020 @04:02PM
(#22561135)
Thank you for the response, I find this very interesting.
So is it true to say that the production of a particular solution to a differential equation is a statement about how a system will behave based on its initial conditions, and that the statement captures within it the principle of action minimization by virtue of the fact that it is a derivation of information from natural laws?
Re:Lagrangian Neural Networks
by whatshisface on Thursday March 12, 2020 @05:22PM
(#22561892)
"Action is minimized" and "F = ma" are the same in the sense that x+5=0 and x+6=1 are the same. In both cases you can go from one to the other with a sequence of deductions. There's no way to tell which one is the "natural law" because you can take either one as a fact and derive the other as a consequence.
Re:Lagrangian Neural Networks
by Brain_Thief on Friday March 13, 2020 @05:36PM
(#22571074)
Interesting - your response makes sense. Thanks again for the information.
I just assume physicists are weirdos like that, they want reality to do easy and nice things, not math :)
As you see it clicks for many people, and doesn't mean much for others. Back then it was a very welcome click.
After all, it should be possible to find antiderivatives and solve ODEs without doing the dance with substitutions and change of variables, but still, we find it many magnitudes easier than just looking at it and divining the correct solution.