News for hackers. Stuff that matters.
Re:Transformer learning explained: Coinductive guide to indu... by frbd on Tuesday February 28, 2023 @11:50AM (#34971547)
The Geometric Deep Learning theory developed by Bronstein ea. is also a very nice unifying theory for deep learning, using Graph Neural Networks. Did you study the interactions with these?
Re:Transformer learning explained: Coinductive guide to indu... by adamnemecek on Tuesday February 28, 2023 @11:54AM (#34971602)
I did, yes, and there is good amount of overlap. In fact, I will be speaking with one of the authors later this week.

However, I find that my approach relies on fewer concepts. There are some concepts that our approaches share but what I like about my approach is that all the concepts are just parts of the Hopf algebra whereas in GDL, they are somewhat ad hoc.

My approach also ties nicely with other things like linear logic and PL theory and therefore provides concrete steps on how to build the next gen machine learning framework which might very provide verifiable machine learning.

My approach fundamentally says that transformer models, diffusion modes and convnets can be expressed with Hopf convolution which is a generalization of the standard convolution.

Also my approach provides a new learning approach which does away with backwards pass and replaces it with something resembling feedback like the one found in electronic circuits.

Re:Transformer learning explained: Coinductive guide to indu... by niemandhier on Tuesday February 28, 2023 @02:37PM (#34973782)
The book you mention is quiet easy to read and didactically structured, the paper was hard to follow.

Just to encourage people to read the book, even if they did not grokk the paper.