News for hackers. Stuff that matters.
Re:Deficient executive control in transformer attention by roenxi on Thursday June 11, 2026 @03:11AM (#48487261)
Aren't transformers universal function approximators? It seems pretty easy to see executive function as a simple computation. So it would be trivially true that a sufficiently large transformer could model executive function because it could approximate [current transformer] + [an approximation of the executive function algorithm] + [whatever bloat is needed to store state in a transformer].

It seems hard to come up with an argument that executive function can't possibly be approximated with an algorithm. Executive function is basic once the clustering into objects part of the process is done. The only real questions are whether a transformer of sufficient scale is feasible on current hardware and if the engineers with access to the hardware have figured out what to train for yet.

Re:Deficient executive control in transformer attention by fc417fc802 on Friday June 12, 2026 @07:25AM (#48502725)
> whether a transformer of sufficient scale is feasible on current hardware

Just because a particular approach (such as a neural network or transformer) can approximate something doesn't necessarily mean it can do so efficiently. However I share what I infer to be your suspicion that executive function can in fact be easily modeled. I think it's likely to develop on its own depending on the training methodology used.