The LLM we are discussing here does have persistent memory, because OpenAI gave it persistent memory.
"It's literally the same thing as running a random .exe from the internet"
I'm not sure what the point is you're making with that, since downloading a random .exe from the Internet is clearly a security issue. By your own analogy, this is also a security issue. The difference is that OpenAI is doing it for you, you're just using OpenAI's program in the way it was intended to be used.
* most people will find it surprising that showing a photo from the internet to ChatGPT is as unsafe as opening a random, untrusted exe.
* many people don't even understand that it's unsafe to open random, untrusted exes.
Are you seriously suggesting that we should leave all these people to the wolves, because they're less knowledgeable about security vulnerabilities than you?
If you have decided to give a 3rd party control over your LLM context, that's on you. Of course the 3rd party has as much control over the LLM as you do.
It's literally the same thing as running a random .exe from the internet. Of course this can be useful, the .exe could provide a useful function, alternatively it could also steal your data. But you chose to run the .exe. Similarly automating your LLM context generation can be useful, but with exactly the same caveats, whoever influences your LLM context controls the LLM. If you enable persistent memory you give them this control.