Re:The bigger-is-better approach to AI is running out of road
by regularfry on Wednesday June 28, 2023 @02:12PM
(#36510337)
Except that incorrect moves don't imply an incorrect representation.
The internal representation was a literal 8x8 grid of piece locations they could externally change and have it generate moves consistent with the changed position. It's about the clearest example of a learned higher-level internal representation I can think I've seen.
The fact that it didn't also perfectly learn the rules while it was doing that is entirely uninteresting.
Hardly, decoding “Bob picked up an Apple.” requires you to make an association between the token Bob and the token Apple. But that doesn’t imply a LLM is operating above the token level.
And again it failed to make strictly correct moves because its internal representation of the game state was flawed. Objectively an LLM when specifically trained on Othello a game with very simple and completely mechanical rules failed to abstract what those rules actually where.