Re:Let's Think Dot by Dot: Hidden Computation in Transformer...
by imtringued on Sunday April 28, 2024 @03:49AM
(#40186828)
Given an algorithm that takes 100 iterations, ask the computer to perform it in ten iterations. It gives you a nonsense answer. Tell it to do it in 100 steps and it might just be able to do it. What this tells us is that context size appears to be a limiting factor as models get bigger and bigger.
[ Parent
| on HN ]
There are no comments on this comment yet.