News for hackers. Stuff that matters.
Re:A global workspace in language models by DoctorOetker on Tuesday July 07, 2026 @01:40AM (#48814036)
it makes you wonder if it may be more efficient to spend all the weights on one layer, and have a repeating stack of the same layer, one would presume this axis has already been explored with metaparameter sweeps?
Re:A global workspace in language models by smallerize on Tuesday July 07, 2026 @05:12PM (#48823855)
That's called a recurrent neural network (RNN).
Re:A global workspace in language models by DoctorOetker on Tuesday July 07, 2026 @05:18PM (#48823939)
distinction is that GPT allows a lot of parallel computation compared to RNN, but I see how your remark indicates a convergence towards RNNs indeed.