News for hackers. Stuff that matters.
Re:Transformers struggle with generalizing tasks beyond pre-... by jjtheblunt on Tuesday November 07, 2023 @12:27PM (#38179824)
Interesting. Thanks for the comment. What if you ask it to add numbers with more digits?
Re:Transformers struggle with generalizing tasks beyond pre-... by GaggiX on Tuesday November 07, 2023 @12:38PM (#38180005)
The model is trained with a fixed number of tokens, I think it would cause problems with the sinusoidal embeddings layer.