News for hackers. Stuff that matters.

A comment on What Happened to Bert and T5?. Parent comment

Re:What Happened to Bert and T5? by thomasahle on Friday July 19, 2024 @05:56PM (#41011802)
If you look at the classical [transformer architecture picture](https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc...) there is an "encoder" tower on the left and a "decoder" tower on the right.

- Bert is encoder only.

- GPT is decoder only.

- T5 uses both the encoder and the decoder.

[ Parent | on HN ]

There are no comments on this comment yet.