News for hackers. Stuff that matters.

A comment on Attention Sinks in LLMs for endless fluency.

Re:Attention Sinks in LLMs for endless fluency by spidersouris on Monday October 09, 2023 @02:37PM (#37823652)
The paper published by Xiao et al. (2023)[0] states that "a surprisingly large amount of attention score is allocated to the initial tokens, irrespective of their relevance to the language modeling task" (p. 2). Does that mean that task prefixes used for LLM generation (e.g. "translate: [sentence]") are actually attention sinks? Or are they not? I don't really understand what they mean by "irrespective of their relevance to the language modeling task."

[0] https://arxiv.org/pdf/2309.17453.pdf

[ Parent | on HN ]
Re:Attention Sinks in LLMs for endless fluency by cubie on Monday October 09, 2023 @04:54PM (#37825365)
By "irrespective of their relevance to the language modeling task", the authors mean that the semantic meaning of the tokens is not important. These 4 tokens can be completely replaced by newlines (i.e. tokens with no semantic meaning), and the perplexity as measured on a book of 65k tokens is nearly unaffected.

The clue is really that these tokens are just used to "offload" attention scores - their semantic meaning is irrelevant.

[ Parent | on HN ]