Re:When Math Gets Impossibly Hard
by theemathas on Thursday September 17, 2020 @01:36AM
(#24501544)
The parent comment is talking about solving differential equations. That's a very different thing from computing differentials. Automatic differentiation doesn't help with differential equations at all.
Yeah, this was a while back. The guy used to work for the South African military during apartheid and later worked on systems in Canada and Asia. I'm sure you're right that digital computers could do the job today, but back then the software and hardware wasn't close to being up to the task.
Re:When Math Gets Impossibly Hard
by ampdepolymerase on Wednesday September 16, 2020 @11:28PM
(#24500889)
Doesn't literally every optimisation and ML class teaches it nowadays?
Re:When Math Gets Impossibly Hard
by jiggawatts on Thursday September 17, 2020 @12:32AM
(#24501202)
Actually, no! You would think so, but again, it's a boring numerical technique so it is all too often sidelined in favour of the more "flashy" symbolic techniques.
An awful lot of tertiary education has been caught up in the publish-or-perish mentality. Unfortunately for AD, it is not "impressive". Differentiating hideously difficult expressions symbolically is impressive. Publishing a paper packed full of enormous, scary looking expressions is impressive. Merely saying that "we used AD and it worked just fine" is unimpressive.
For decades now we've been incentivizing intellectual wankery over practical utility.
Re:When Math Gets Impossibly Hard
by lambdatronics on Thursday September 17, 2020 @02:01AM
(#24501672)
Here's an example from my field: "The derivatives of the finite-build coils are computed -- with almost no additional effort from the authors -- using automatic differentiation."
Well, what kind of scientist are you, if you are too lazy to take derivatives? /sarcasm
It's simply criminal how underutilised the technique of Automatic Differentiation is! It can solve incredibly complex differentials to machine precision, good numerical stability, and minimal tuning or hand-holding.
Lots of these "difficult" problems aren't, it's just that people aren't aware of the solution because it's not commonly taught in a University setting. Automatic Differentiation is one of those techniques that's "boring" and "numerical", so there are no courses that teach it.