News for hackers. Stuff that matters.
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by newhouseb on Friday May 12, 2023 @09:10AM (#35915701)
Nope, a decoder only transformer is a variant of the original architecture proposed by Google [1]. All variants of GPT that we know about (1 through 3) all roughly use this same architecture which takes only the decoder stack from the original Google paper and drops the encoder [2]

[1] Original Google Paper - https://arxiv.org/abs/1706.03762

[2] Original GPT Paper - https://s3-us-west-2.amazonaws.com/openai-assets/research-co...

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by flangola7 on Friday May 12, 2023 @12:42PM (#35918642)
How can it work without an encoder?