Re:Deficient executive control in transformer attention
by fc417fc802 on Friday June 12, 2026 @07:25AM
(#48502725)
> whether a transformer of sufficient scale is feasible on current hardware
Just because a particular approach (such as a neural network or transformer) can approximate something doesn't necessarily mean it can do so efficiently. However I share what I infer to be your suspicion that executive function can in fact be easily modeled. I think it's likely to develop on its own depending on the training methodology used.
It seems hard to come up with an argument that executive function can't possibly be approximated with an algorithm. Executive function is basic once the clustering into objects part of the process is done. The only real questions are whether a transformer of sufficient scale is feasible on current hardware and if the engineers with access to the hardware have figured out what to train for yet.