Re:Smallest transformer that can add two 10-digit numbers
by silentvoice on Saturday February 28, 2026 @05:24PM
(#47200951)
I have no submission for this but I joined the hype in my own way by optimizing the training loop. These tiny models are not really well suited to frameworks like pytorch, and with highly patient AI agents we can now just inline the whole thing into C++ just to see what happens, which I do below:
https://www.reidatcheson.com/transformer/llm/ml/cuda%20graph...