News for hackers. Stuff that matters.
Re:Scaling Transformer to 1M tokens and beyond with RMT by Robotbeat on Monday April 24, 2023 @11:51AM (#35689173)
It’s not literally the same as the base model training, but you ARE literally training feed forward networks when you do this in the prompt. That’s how transformers work.

There are no comments on this comment yet.