I think the medium, data collection, and scale matter. It's never been so affective or efficient as it is now (and will become).

Gathering data from a million people on which shade of red makes them more likely to click a button is entirely different today due to the scale, how cheap it is to setup, and how cheap it is to tweak. This data can then be used to "nudge" people towards a direction that you benefit from (and they may or may not benefit from, and society at large may or may not benefit from). At scale, these very small nudges can have an impact. The unregulated methods we use for this keep improving (AI).

Not to throw shade, but there's a reason why Amazon has been hiring behavioural psychologists. We should be aware and thinking about this.

I disagree in that I feel the higher effectiveness results in better UI.

Perhaps we need to simultaneously inform people through better education at the same time for how to resist the urge to spend borrowed money whenever possible?

Incentives of the publisher and the consumer aren't always aligned. Publisher might want you to spend / use the mobile app (tracking) / budge your political leaning / confuse you with disinformation / etc. The consumer / user is simply outgunned, and it's getting more and more lopsided. Regulation is inevitable.

This isn't just about good UI. Not everyone is using these sort of behavioural tests to present a better UI. It's also about influence (micro influence). I'm not sure you're seeing the whole picture.

It sounds like conspiracy, but Obama and Cameron had "Nudge Units", and that was 5 years ago.

http://freakonomics.com/podcast/white-house-gets-nudge-busin...

https://www.warc.com/newsandopinion/opinion/how_the_likes_of...

https://www.forbes.com/sites/beltway/2015/09/16/obama-nudge-...