News for hackers. Stuff that matters.
Re:LLM Powered Autonomous Agents by zurfer on Tuesday June 27, 2023 @05:33AM (#36490766)
We probably stay with the next token prediction task for training the foundation. It generalizes really well.

However, transformers are not able to loop. In a fixed amount of computing steps, they need to predict. For long-term planning, we probably want something more recursive/iterative (but that will be harder to train).

Also today we can compensate that with wrappers (eg. Langchain), but ultimately the machine will learn end to end.

There are no comments on this comment yet.