Something not mentioned here is advertising and ad tech. With traditional search advertising going away, psyads - psychologically manipulative advertising that changes answers and text generations to convince people to buy and do things - will become the new nightmare.

In the same way language models can mimic any character ("write a poem in the style of Jar Jar Binks") they can also mimic voice and tone in a way designed to appeal - and sell to - an individual based on their personal data and history.

ChatGPT and its clones require login and often credit card and phone number, they collect your data, and they feed everything you do back to language models for training. I'm only aware of one that has a privacy policy that would prevent this.

Caveat Scrutator