News for hackers. Stuff that matters.
Re:A global workspace in language models by fc417fc802 on Monday July 06, 2026 @08:56PM (#48812428)
Because padding the output in such a manner (which keep in mind is immediately fed back as input) has been shown to increase model performance. It's one of the many reasons to question what "thinking" traces are really doing.

If you consider how the attention mechanism works then a very hand wavey intuition is that despite being entirely arbitrary additional tokens should still provide the opportunity for additional information processing.

There are no comments on this comment yet.