Re:Scaling Transformer to 1M tokens and beyond with RMT
by Robotbeat on Monday April 24, 2023 @11:51AM
(#35689173)
It’s not literally the same as the base model training, but you ARE literally training feed forward networks when you do this in the prompt. That’s how transformers work.
[ Parent
| on HN ]
There are no comments on this comment yet.