News for hackers. Stuff that matters.
Re:Predictive coding has been unified with backpropagation by 0lmer on Monday April 05, 2021 @04:25PM (#26704203)
But does predictive coding perceived as a valid theory for cortical neurons functioning? There was a paper from 2017 drawing similar conclusions about backprop approximation with Spike-Timing-Dependent Plasticity: https://arxiv.org/abs/1711.04214 Looks more grounded to current models of neuronal functioning. Nevertheless, it changed nothing in the field of deep learning since then.
Re:Predictive coding has been unified with backpropagation by jwmullally on Monday April 05, 2021 @06:36PM (#26705523)
Some general background on STDP for the thread:

Biological neurons don't just emit constant 0...1 float values, they communicate using time sensitive bursts of voltage known as "spike trains". Spiking Neural Networks (SNN) are a closer aproximation of natural networks than typical ML ANNs. [0] gives a quick overview.

Spike-Timing-Dependant-Plasticity is a local learning rule experimentally observed in biological neurons. It's a form of Hebbian learning, aka "Neurons that fire together wire together."

Summary from [1]. The top graph gives a clear picture of how the rule works.

> With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to Long-Term Potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to Long-Term Depression (LTD) of the same synapse.

---

[0]: https://towardsdatascience.com/deep-learning-versus-biologic...

[1]: http://www.scholarpedia.org/article/Spike-timing_dependent_p...

Re:Predictive coding has been unified with backpropagation by andyxor on Monday April 05, 2021 @04:55PM (#26704610)
as long as the model requires delta rule, or 'teacher signal' based error correction it is not biologically plausible.