I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately

this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations.

I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.