News for hackers. Stuff that matters.
Re:Scaling Transformer to 1M tokens and beyond with RMT by nl on Monday April 24, 2023 @09:10PM (#35695365)
Yes, this.

There's clearly an internal representation of the relationships that is being updated.

If you follow my Twitter thread it shows some temporal reasoning capabilities too. Hard to argue that is just copied from training data: https://twitter.com/nlothian/status/1646699218290225154

There are no comments on this comment yet.