Re:Scaling Transformer to 1M tokens and beyond with RMT
by nl on Monday April 24, 2023 @09:07PM
(#35695344)
That's not the same thing at all.
It absolutely needed to know who the successor would be via training data.
But to know that "The Queen of England died" also means that the head of state of Australia has changed means that it has an internal representation of those relationships.
(Another way of seeing this is with multi-modal models where the visual concepts and word concepts are related enough it can map between the two.)
It absolutely needed to know who the successor would be via training data.
But to know that "The Queen of England died" also means that the head of state of Australia has changed means that it has an internal representation of those relationships.
(Another way of seeing this is with multi-modal models where the visual concepts and word concepts are related enough it can map between the two.)