Re:Transformer Attention is off by one
by notanothereric on Tuesday July 25, 2023 @12:27AM
(#36857889)
If the author just proposed his new function neo_max() and quoted The Architect from The Matrix everyone here would be enjoying the April Fools Day joke for 2024…
“ Your life is the sum of a remainder of an unbalanced equation inherent to the programming of the matrix. You are the eventuality of an anomaly, which despite my sincerest efforts I have been unable to eliminate from what is otherwise a harmony of mathematical precision. While it remains a burden assiduously avoided, it is not unexpected, and thus not beyond a measure of control. Which has led you, inexorably, here.”
“ Your life is the sum of a remainder of an unbalanced equation inherent to the programming of the matrix. You are the eventuality of an anomaly, which despite my sincerest efforts I have been unable to eliminate from what is otherwise a harmony of mathematical precision. While it remains a burden assiduously avoided, it is not unexpected, and thus not beyond a measure of control. Which has led you, inexorably, here.”
Hence ergo - you are a NaN. A divide-by-zero.