Re:John Carmack and Rich Sutton partner to accelerate develo...
by og_kalu on Saturday October 07, 2023 @05:08PM
(#37805589)
LLMs already do this in their own way. It's not like they learn to predict through magic osmosis. They make predictions and depending on how right or wrong they are, their neurons are shifted by grad descent to accommodate this error. Then they predict again.
The Text is the environment, The Text is the world and they are very much interacting with it.
Re:John Carmack and Rich Sutton partner to accelerate develo...
by bwv848 on Sunday October 08, 2023 @03:42AM
(#37808603)
Not really, LLM Decoders(without RLHF) are usually trained by teacher forcing, that means they just get learning singals from just predicting one token.
The Text is the environment, The Text is the world and they are very much interacting with it.