Self-Rewarding Language Models

Re:Self-Rewarding Language Models by gary_0 on Saturday January 20, 2024 @08:01PM (#39074331)

The rules of Go are not exactly "programmed in" to AlphaGo; they are only provided so the model knows what the valid moves and win conditions are as it self-trains. The latest versions of the model can be quickly adapted to play other games[0] and MuZero can "master games without knowing their rules"[1]. Also, some of DeepMind's earliest work was a deep reinforcement learning model that could play seven different Atari games with no adjustments needed to switch games[2].

[0] https://en.wikipedia.org/wiki/AlphaGo_Zero

[1] https://en.wikipedia.org/wiki/MuZero

[2] https://arxiv.org/abs/1312.5602

[ Parent | on HN ]

Re:Self-Rewarding Language Models by modeless on Saturday January 20, 2024 @08:25PM (#39074502)

The rules of Go very much are programmed in to AlphaGo. The neural nets are generic, sure. But the game rules programmed manually into the training system are vital to the system working. Even with MuZero: while it uses a world model which is learned, ultimately during training there is still grounding in the manually programmed rules of whatever game it's playing. When it trains on Atari, the Atari system (manually programmed, obviously) provides the grounding rules.

What I am saying is there is no analogous source of manually programmed rules to ground language models during training.

[ Parent | on HN ]