News for hackers. Stuff that matters.
Re:Automatic Differentiation with Julia by IngoBlechschmid on Tuesday October 23, 2018 @10:31AM (#18283662)
Yes, definitely, there are even battle-tested implementations for Fortran available.

Though I have never seen AD frameworks used in the production contexts for neural networks/backpropagation. As you say, the code for this seems to be mostly handrolled. Please take this negative statement with a grain of salt, I don't actually work in machine learning.

Re:Automatic Differentiation with Julia by kxyvr on Tuesday October 23, 2018 @11:07AM (#18283980)
Backpropagation is literally the reverse mode AD applied to neural nets. That said, I wish had a paper that I could point to that shows explicitly that. From what I can tell, the AD community has known this basically since the start, but for whatever reason backpropogation is taught rather than the more general technique.