News for hackers. Stuff that matters.
Re:Auto-Regressive Next-Token Predictors Are Universal Learners by nonrandomstring on Saturday September 23, 2023 @04:37AM (#37621613)
Yeah I immediately thought, isn't this congruent to a statement about Turing machines. Sure there are classes of many things that are computationally equivalent, including computers made of paper-tape, tin cans and string. Just most of them are horrendously inefficient and useful only as thought experiments.

I saw this again in audio synthesis but with more nuance. Most methods are equivalent in some crazy limit, but all have a "special" area of most useful effectiveness. For example in theory you can predict a signal of many minutes or hours just using linear prediction (LPC), but only at the cost of a gargantuan parameter space that's less efficient than just sampling the signal.

Nonetheless it is nice to see that researchers are connecting up these dots, even if the pure maths behind it isn't saying anything obviously useful right away. Who knows what insights this might lead to for discovering other new methods of computation.

There are no comments on this comment yet.