News for hackers. Stuff that matters.

A comment on Transformer learning explained: Coinductive guide to inductive transformer heads.

Re:Transformer learning explained: Coinductive guide to indu... by uoaei on Tuesday February 28, 2023 @04:44PM (#34975285)
You comment elsewhere that the Hopf coherence-based optimization procedure works locally. How does it compare to local predictive coding techniques?

E.g.: https://openreview.net/pdf?id=PdauS7wZBfC

[ Parent | on HN ]
Re:Transformer learning explained: Coinductive guide to indu... by adamnemecek on Tuesday February 28, 2023 @05:15PM (#34975681)
I have to read the paper in more detail but I imagine there's a good amount of overlap.
[ Parent | on HN ]