News for hackers. Stuff that matters.
Re:OpenAI – transformer debugger release by bilsbie on Tuesday March 12, 2024 @07:21AM (#39678245)
How many transformers are in a typical LLM? Or is the whole thing considered a transformer?
Re:OpenAI – transformer debugger release by jerpint on Tuesday March 12, 2024 @08:10AM (#39678543)
The architecture as a whole is referred to as a transformer (autoregressive decoder-only architecture in the case of gpt style LLMs).

Note that there can be other types of LLMs too that are not necessarily transformer based (SSM, RNN, etc)

Re:OpenAI – transformer debugger release by dfgtyu65r on Tuesday March 12, 2024 @07:27AM (#39678278)
Normally, the LLM is composed of multiple transformer blocks, where each block consists of the (mutli-head) attention and fully-connected feedforward components. These are then stacked on top of each other to give the final output of the network.