> How do you think a transformer model learns?

Sorry I don't know much about transformers, it's why I was reading! My small understanding is that transformers allow a kind of 'differentiable indirection' that has become practical to train now that we have these modern GPUs and automatic differentiation frameworks coupled with training algorithms.

You are right but here's the kicker: they learn even without autodiff! So my paper tries to explain that. I argue that Hopf coherence is a better autodiff.

You should peep the Pang2014 paper, I think you will understand better what's happening.

Is it possible to explain what you are doing like I'm a normie who doesn't know and isn't willing to learn algebraic geometry, category theory, or control theory? In other words, explain only in terms of plain finite dimensional linear algebra like eigenvalues, left- and right-eigenvectors, row- and column-stochastic matrices, conjugate transpose, trace, matrix inverse, matrix exponential, etc.?

Hopf algebras are analogous to electrical circuits with feedback and incrementally "learn" eigenvalues by processing incoming data without needing additional processing.