News for hackers. Stuff that matters.

A comment on Predictive coding has been unified with backpropagation.

Re:Predictive coding has been unified with backpropagation by adamnemecek on Monday April 05, 2021 @02:13PM (#26702462)
I think that this sort of forward backward thing is a very general idea. There’s a one to many relationship called the adjoint, and a many to one relationship called the norm.

I wrote something about this here https://github.com/adamnemecek/adjoint

[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by tsmithe on Monday April 05, 2021 @02:42PM (#26702865)
In fact, the compositional structure underlying that of predictive coding [0,1] is abstractly the same as that underlying backprop [2]. (Disclaimer: [0,1] are my own papers; I'm working on a more precise and extensive version of [1] right now!)

[0] https://arxiv.org/abs/2006.01631 [1] https://arxiv.org/abs/2101.10483 [2] https://arxiv.org/abs/1711.10455

[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by eli_gottlieb on Monday April 05, 2021 @04:02PM (#26703892)
Hurry and publish before I have manuscripts ready applying these results.
[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by tsmithe on Monday April 05, 2021 @04:25PM (#26704210)
Hey, Eli :-)

I'm working on it; I'll send you an e-mail. Things quickly turned out to be more general than I realized last year.

[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by eli_gottlieb on Monday April 05, 2021 @08:14PM (#26706406)
That would make sense. The whole ACT/categorical cybernetics community has been working out how massively general optics are :-).
[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by selimthegrim on Monday April 05, 2021 @03:28PM (#26703477)
What were you going to say about Young tableaux?
[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by adamnemecek on Monday April 05, 2021 @03:36PM (#26703578)
Dynamic programming and reinforcement learning are just diagonalizations of the Young tableau. This is related to the spectral theorem.
[ Parent | on HN ]
Re:Predictive coding has been unified with backpropagation by selimthegrim on Tuesday April 06, 2021 @04:16PM (#26716835)
Interesting. Is there some concrete realization you had in mind?
[ Parent | on HN ]