Re:Want to get started with LLMs? Here's what you need to know
by simonw on Friday October 13, 2023 @01:17PM
(#37873022)
There's an interesting new paper about this problem: https://arxiv.org/abs/2310.02226
"Think before you speak: Training Language Models With Pause Tokens"
Basic idea is to teach the LLM to occasionally insert a "pause" token, which outputs nothing but gives it a chance to perform another round of operations on the way to the answer.
Of course, if we'd want to generalize it, we should have a structure where the model itself can actually make the decision to continue work for one more iteration, and also ensure that it can retain some of the temporary work-in-progress notes (currently it can see only what the previous iteration output as part of the answer).