News for hackers. Stuff that matters.
Re:From scratch: reverse-mode automatic differentiation (in ... by a1369209993 on Monday June 15, 2020 @03:00AM (#23524567)
Technically there is:

  D f = f' <=> S f' = f  # inverse
  
  # chain rule
  D (f . g) x = D f (g x) * D g x
  # divide by D g x
  D (f . g) x / D g x = (D f) (g x)
  # S vs D inverse
  D (S f' . g) x / D g x = (f' . g) x
  # commute = and integrate both sides
  S (f' . g) x = S \x( D (S f' . g) x / D g x ) x
  # rename and drop argument
  S (f . g) = S \x( D (S f . g) x / D g x )
The problem is that the RHS is, in the general case, not any less complex (and often much more) than the LHS.

Edit: Note that (S f . g) is integrating only f, then composing with g, which is what you want; it's the S \x(...) that causes problems.

Re:From scratch: reverse-mode automatic differentiation (in ... by IngoBlechschmid on Monday June 15, 2020 @04:16AM (#23525054)
Nice observation, thank you for sharing!