Re:A global workspace in language models
by DoctorOetker on Tuesday July 07, 2026 @01:40AM
(#48814036)
it makes you wonder if it may be more efficient to spend all the weights on one layer, and have a repeating stack of the same layer, one would presume this axis has already been explored with metaparameter sweeps?
[ Parent
| on HN ]
Re:A global workspace in language models
by smallerize on Tuesday July 07, 2026 @05:12PM
(#48823855)
That's called a recurrent neural network (RNN).
Re:A global workspace in language models
by DoctorOetker on Tuesday July 07, 2026 @05:18PM
(#48823939)
distinction is that GPT allows a lot of parallel computation compared to RNN, but I see how your remark indicates a convergence towards RNNs indeed.