Re:LLM Powered Autonomous Agents
by zurfer on Tuesday June 27, 2023 @05:33AM
(#36490766)
We probably stay with the next token prediction task for training the foundation. It generalizes really well.
However, transformers are not able to loop. In a fixed amount of computing steps, they need to predict. For long-term planning, we probably want something more recursive/iterative (but that will be harder to train).
Also today we can compensate that with wrappers (eg. Langchain), but ultimately the machine will learn end to end.
However, transformers are not able to loop. In a fixed amount of computing steps, they need to predict. For long-term planning, we probably want something more recursive/iterative (but that will be harder to train).
Also today we can compensate that with wrappers (eg. Langchain), but ultimately the machine will learn end to end.