News for hackers. Stuff that matters.
Re:Rethinking Attention with Performers by sjg007 on Monday October 26, 2020 @06:47PM (#24902405)
Transformers are kinda similar to state vectors. They are tracking the current state of the world. The input becomes the output which is the input to the next iteration. The transformer transform the input to the output ad infinitum until a stop token is reached.

There are no comments on this comment yet.