Re:Is next token prediction the new "E = mc2"?
by d--b on Thursday December 21, 2023 @07:30AM
(#38719616)
It feels very strange to put "next token prediction" on the same level as "E = mc2".
"next token prediction" is an experiment that's puzzling, while E=mc2 is a theory (that Einstein developed to explain the puzzling experiments that were unexplained at the time).
It may or may not be that in the end there is a "simple formula" that explains why LLMs work, but we're not there yet.
"next token prediction" is an experiment that's puzzling, while E=mc2 is a theory (that Einstein developed to explain the puzzling experiments that were unexplained at the time).
It may or may not be that in the end there is a "simple formula" that explains why LLMs work, but we're not there yet.