Re:Transformers struggle with generalizing tasks beyond pre-...
by GaggiX on Tuesday November 07, 2023 @05:56PM
(#38184242)
I trained my models from scratch, the tokenization is extremely simple and designed just for this task, decoder-only models can learn to do sums of arbitrarily large numbers if the model is large enough (they are computationally bounded). The decoding method used was greedy or simple sampling.
[ Parent
| on HN ]
There are no comments on this comment yet.