News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by Legend2440 on Monday July 24, 2023 @03:42PM (#36853234)
Because until he tries it, who knows if it works?

There are a thousand papers out there making minor tweaks to the transformer architecture. 99% of them are also worthless and forgotten.

Re:Transformer Attention is off by one by debugnik on Monday July 24, 2023 @03:49PM (#36853309)
> Because until he tries it, who knows if it works?

That's precisely what he shared this for, though. So someone willing to train a model with this tweak tries it.