News for hackers. Stuff that matters.

A comment on Σ-GPTs: A New Approach to Autoregressive Models. Parent comment

Re:Σ-GPTs: A New Approach to Autoregressive Models by toxik on Friday June 07, 2024 @01:07PM (#40610614)
This problem formulation has been around for a while, it’s kind of the holy grail of modeling. What is new compared to PixelCNN and related is this position embedding idea.
[ Parent | on HN ]
Re:Σ-GPTs: A New Approach to Autoregressive Models by cs702 on Monday June 10, 2024 @09:36AM (#40633556)
Yes, the modification the authors propose is indeed "simple."

It's "obvious" only in hindsight.

[ Parent | on HN ]