Every night it could go through its conversations and condense them down to a tiny compressed sample leveraging context from previous long term memory. We could call it sleeping.

And fine-tune itself? Fascinating.

Yes and perhaps during this nightly compression and fine tuning, new inferences are made on previous conversations by connecting then with longer-term context and we can call that dreaming

My first thought of this was like a scratchpad, so if you have a chatgpt session there's a place where chatgpt can add/remove text, and it is continuously available as input.

Right now, you can't ask chatgpt to "think of a person, and the user will guess who it is". It can decide to think of something, but has nowhere to put that information. It should be put into the scratchpad.

Scratchpad could potentially be used for solving problems step by step too, much like how we consciously think through problems step by step. If it gets too complicated for us, the mental scratchpad we use is augmented by a physical notepad or other system.

You could also say that it enables an internal monologue.