News for hackers. Stuff that matters.

A comment on Streaming LLM – No limit on context length for your favourite LLM.

Re:Streaming LLM – No limit on context length for your favou... by Trapais on Tuesday October 03, 2023 @02:00AM (#37748418)
Looks like longformer to me. They just renamed "global attention" into "attention sink" and removed silly parts(distilled attention) and BERT parts([CLS] saw all N tokens, there is no need for BOS to see all tokens)
[ Parent | on HN ]

There are no comments on this comment yet.