The backprop idea is one instance of a general idea called reverse accumulation (which you can use in other contexts): http://www.win-vector.com/blog/2010/07/gradients-via-reverse...