Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart
by faramarz on Wednesday April 17, 2024 @04:55PM
(#40069972)
GPT-4 is configurable up to 96 layers, each running their own embeddings. I think it was a business choice to afford the compute while they scale.
[ Parent
| on HN ]
There are no comments on this comment yet.