News for hackers. Stuff that matters.
Re:Memorizing Transformers by visarga on Friday May 20, 2022 @12:58PM (#31450020)
> Wow.

Transformers are very limited in the size of the attention window. They can take a few thousand tokens at maximum. But your data might not fit into the window, and you also don't want to have to fine-tune the model. This paper offers a solution.

There are no comments on this comment yet.