News for hackers. Stuff that matters.
Re:Predictive Coding Approximates Backprop Along Arbitrary C... by orbifold on Tuesday October 06, 2020 @05:20AM (#24695993)
They don't address the fundamental issue of BPTT in an RNN model. To my mind this is a much larger block to biological plausibility than how to track the error locally. Besides this has been addressed in various forms in earlier work:

- Hochreiter and Schmidhuber present a truncated version of BPTT in the original LSTM paper that tracks part of the error locally and forward in time (https://www.bioinf.jku.at/publications/older/2604.pdf)

- Bellec et al. https://www.nature.com/articles/s41467-020-17236-y present a similar algorithm that tracks the error with something they call eligibility traces. Despite the title of the paper the method is also applicable to non-spiking RNNs.

- Bohnstigl et al. present another solution: https://arxiv.org/pdf/2007.12723.pdf

There are no comments on this comment yet.