Re:Ask HN: Can someone ELI5 Transformers and the “Attention ...
by drexlspivey on Wednesday May 17, 2023 @02:20PM
(#35979244)
AFAIK Transformers and context size are orthogonal concepts. You could have large token contexts before. The transformer directs the “attention” to a specific word/token inside the context.
[ Parent
| on HN ]
There are no comments on this comment yet.