Re:Inkling 2 Mini
by NiloCK on Friday November 20, 2026 @09:00PM
(#50585113)
> a looped transformer, running the middle layers of the model three times for every token.
Completely unqualified take here, but unconditional re-runs of a fixed set of layers doesn't strike me as meaningfully 'loopy'. The exact performance profile is available in a more trad architecture by just inserting the layers after one another?
My presumption on loopy architectures was that they looped longer on token selections that were more difficult / higher stakes, with automatic kick-outs after some n loops to avoid machine analysis paralysis.
ALTHOUGH I guess that the presented architecture may also be useful insofar as it can fit in less memory than the same model with a tripled set of , and possibly that training over a looping architecture converges faster / truer than over naive architectures of the same size.
Completely unqualified take here, but unconditional re-runs of a fixed set of layers doesn't strike me as meaningfully 'loopy'. The exact performance profile is available in a more trad architecture by just inserting the layers after one another?
My presumption on loopy architectures was that they looped longer on token selections that were more difficult / higher stakes, with automatic kick-outs after some n loops to avoid machine analysis paralysis.
ALTHOUGH I guess that the presented architecture may also be useful insofar as it can fit in less memory than the same model with a tripled set of , and possibly that training over a looping architecture converges faster / truer than over naive architectures of the same size.