News for hackers. Stuff that matters.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by drexlspivey on Wednesday May 17, 2023 @02:20PM (#35979244)
AFAIK Transformers and context size are orthogonal concepts. You could have large token contexts before. The transformer directs the “attention” to a specific word/token inside the context.

There are no comments on this comment yet.