Arguably the right kind of structure for deciding on what uses LLMs should be put to in its territory is a democratically elected government.
Gacha/paid loot box mechanics are a great example of this. They are user hostile and serve no purpose other than to be addictive.
Mobile apps already employ slews of psychological modeling of individual user's behavior to try and manipulate people into paying money. Freemium games are infamous for letting you win and win, and then suddenly not, and slowly on ramping users into paying to win, with the game's difficulty adapting to individual users to maximize $ return. There are no laws against that, and the way things are going, there won't ever be.
I guess what I'm saying is that sometimes the law lags (far) behind reality, and having some companies go "actually, don't use our technology for evil" is better than the alternative of, well, technology being used for evil.
No, I can honestly say that I do not lose any sleep over this, and I think it's pretty weird that you do. Humans have been fending off human advertisers and scammers since the dawn of the species. We're better at it than you account for.
https://www.nbcnews.com/business/consumer/people-are-losing-...
The data says we are not that good and getting 30% worse every year.
https://www.pbs.org/newshour/economy/new-federal-estimate-fi...
US tax fraud is estimated to be $1 trillion a year
https://www.latimes.com/business/story/2021-04-13/tax-cheats...
https://cybersecurityventures.com/hackerpocalypse-cybercrime...
Huge numbers of people are absolutely terrible at it and routinely get rinsed out like rags.
We have already seen that users can become emotionally attached to chat bots. Now imagine if the ToS is "do whatever you want".
Automated cat fishing, fully automated girlfriend scams. How about online chat rooms for gambling where half the "users" chatting are actually AI bots slowly convincing people to spend even more money? Take any online mobile game that is clan based, now some of the clan members are actually chatbots encouraging the humans to spend more money to "keep up".
LLMs absolutely need some restrictions on their use.