They don't need that much handholding. They are a couple memory augmented gpt papers out now (self-refine, reflexion etc). This is by far the most involved in terms of instructing memory and reflection.

It helps for control/observation but it is by no means necessary.