The blasé attitude of the people building this technology is what makes it horrifying. Literally the FIRST major objections I heard to generative AI/deepfakes was the strong possibility if not guarantee of how it would be utilized by bad actors, be they interpersonal or political in nature. And over and over the tech sphere has downplayed and ignored this reality in favor of the ephemeral "progress" which also happens, by sheer coincidence I'm sure, to be their paths to success both professionally and financially. And on this progress marched until we've reached a place where it is now feasible that a high schooler could take the likeness of a girl he likes, shove it into an image generator, and make porn out of her without her consent or knowledge and be totally free to distribute that however he sees fit.

THAT is horrifying. It's horrifying both by the base actions at play and, far moreso IMO, that because someone in the tech space thought it was monetizable enough to warrant investment, it will happen. It did happen. That genie is now out if it's box and when confronted with the ethical dilemmas it seems most people in the engineering space are content to shrug their shoulders and go "shit happens."

That's horrifying. It's horrifying that all a technology has to do is demonstrate it's ability to make money, no matter how socially or culturally corrosive it might be, no matter how much harm it might do to individuals or even entire professions, no matter how unethical and just straight up creepy it is, doesn't matter. If it will make someone rich, it will be done, and it will be done with the best hardware and expertise money can buy, and that social order will be corroded, that culture will be corroded, those individuals will be harmed, that profession will be obliterated, and those checks will be cashed.

THAT'S horrifying.