> People who can handle manipulative messages from ads should be mostly resistant to manipulative messaging from bots.

Not necessarily. LLMs at the model of GPT-4, and eventually above, affect the economics of advertising. Currently, there's only so much sense in optimizing your ads before it's cheaper to just spam more of it (er, "increase exposure"). LLMs will make it cheaper to optimize the ads further (i.e. get more bang for the same buck), so I expect the baseline manipulative effect of most ads to increase.

Why work on a broad message (ad) when you can afford (cpu time wise) to have a dedicated con artist target every individual in the world at the same time?

Exactly, and thanks for making this point explicitly.

It's one of the end-game consequences of the change in advertising economics I mentioned: with costs of future AI models going down, eventually companies may be able to afford to generate individually personalized and highly persuasive ads on the fly. A personal dedicated con artist for everyone, indeed.

What's more these people often don't know it's even happening.

In fact, the odds are extremely high that you, me, and everyone else here has been taken in by some kind of ad or ad-like rhetoric and internalized it as just being normal world-view.

If there exist those who haven't even been fooled in such a way, they must certainly have the self-awareness to nevertheless entertain doubts on the matter.

Yes, everyone has to one degree or another. No one is an expert in all fields, no one has perfect knowledge and so you will often be presented with things that seem perfectly reasonable but which are entirely false. Worse yet, most of the information you receive will be partly true and partly false with no easy way to tell which is which without engaging in a significant amount of legwork. The trope is the Gell-Mann Amnesia effect which you face every morning when you read a few interesting articles or listen to the daily news. The practices of active deception for sales and influence are all quite clearly laid out by Edward Bernays in his book Propaganda, even our modern concept of "influencers" was known during his day, the more exoteric framing being that people will trust and follow the advice of people that they like or think are similar to them in some way.

I have a new law: "Sufficiently large quantitative change always turns into qualitative change."