News for hackers. Stuff that matters.
Re:Let's Think Dot by Dot: Hidden Computation in Transformer... by imtringued on Sunday April 28, 2024 @03:49AM (#40186828)
Given an algorithm that takes 100 iterations, ask the computer to perform it in ten iterations. It gives you a nonsense answer. Tell it to do it in 100 steps and it might just be able to do it. What this tells us is that context size appears to be a limiting factor as models get bigger and bigger.

There are no comments on this comment yet.