News for hackers. Stuff that matters.
Re:Transformers struggle with generalizing tasks beyond pre-... by GaggiX on Tuesday November 07, 2023 @05:56PM (#38184242)
I trained my models from scratch, the tokenization is extremely simple and designed just for this task, decoder-only models can learn to do sums of arbitrarily large numbers if the model is large enough (they are computationally bounded). The decoding method used was greedy or simple sampling.

There are no comments on this comment yet.