Forget celebrity deepfakes, I for one want to see a Lindsey Graham and Felipe VI crossover Dune porn parody (who’s got the biggest sandworm). It’d reinvent the concept of “gay for pay.”

Funny to think about but in no way is this a realistic solution

And then immediately jumped to “lol you’re gay” as something the girls should use against the guys

This is just casually dropped as though a photo-realistic AI image == an amateur painting, which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy.

> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels.

Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world.

And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.

You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.

Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.

This statement is hilarious because we cannot even do the former for schoolchildren, let alone the latter. Between casual bullying resulting in fatalities due to the utter disinterest of the administration at play, to school shootings where police cower outside while someone murders children en masse.

Apparently being a child now is just grounds to have people make pornography of you, bully you relentlessly, with the distinct possibility you might die in a hail of gunfire one day for literally no reason.

No wonder the birth rate has cratered.

> You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings

CAN and SHOULD. INCREDIBLY illegal. Like registering the creator as a sex offender illegal, because that's what this is. It's a sex crime.

> but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.

So we just give up?

That said, in a global internet any technology that can be created, will be crated, and made free and open for anyone to use. We can never escape this fact, and can only educate people on how to protect themselves as individuals as much as possible.

Google, Facebook, and modern surveillance capitalism app culture is -optional-. Smartphones have turned into a net negative that make most people miserable for instance. I for one do not have a smartphone, and am happier for it.

Maybe, but that's not an excuse to do it, like: "I do it because anyway someone else will". Silicon Valley is throwing a lot of money into stuff that "someone else" would do, and it is not clear to me that this someone else would always have that kind of resources.

I think it should be legal to burn flags. I would also never do it or advise anyone to do it.

Sometimes a free country requires allowing assholes to exist, and educating each other on how to ignore them.

I'm glad you asked!

Step 1: Anyone who uses any media, be it watercolors or generative AI, to create pornographic content of a minor should go to a LOT of prison and be registered as a sex offender.

Step 2: Educate children on these tools, what they may be used for, what they may not be used for, and illustrate vividly the extreme consequences thereof.

Step 3: When it inevitably happens anyway, enforce those consequences. The offender needs to go away for a long time, and be subject to rehabilitation programs to re-enter society. They should be treated as any other sex offender would, because that's what they are. Their age does not change their crime.

Step 3.5: Support the victim. Make it abundantly clear that anyone possessing this material any further will be subject to less severe certainly, but still severe consequences. It is to be deleted, to the maximum extent possible, from the maximum number of places possible.

Step 4: Any platform that permits this content to be distributed upon it should also be subjected to extremely steep fines and if the issue persists, seizure by the appropriate authorities. And frankly this should be expanded to all CAM, regardless of origin. Don't wanna regulate it, or feel you can't enforce this? Then shut down. Nobody asked you to to build a website and if you can't keep CP off your website, then your website doesn't need to exist, full stop.

Step 5: At this exact moment, begin changing how these generative tools are constructed. Make sure they only use training materials that are properly licensed and given over by their subjects and creators. If a given AI contains the ability to generate pornographic material, then it's distribution should also, in turn, be regulated as pornography already is. Yes, that will be probably be just as ineffective as it is for regular pornography. No, that is not an excuse to not do it. A tool that can make pornography on demand is indistinguishable from pornography.

And yes I am certain all of this will stifle innovation in the AI space. I don't care. Frankly if this kills it off entirely I think that's an unmitigated win.

Do you really suggest we abolish the right to privacy and anonymous speech in order to support the type of universal censorship you propose?

No, we can't stop people sharing it over encrypted channels. What we can do is mandate that these AI include identification data in their generated images, which is tied to a singular person. And from there, we can identify and prosecute the creator, and through court orders to whatever platform holder is at play, get a list of everyone who downloaded it and shared it.

We do this shit constantly for major rights holders. We know how. We just choose not to because too many people think it's "just boys being boys" and enable a culture that thinks sex offenses aren't worth our time.

You talk of punishment in the context of physical harm (fist vs. face), while deepfakes concern mental distress at worst. 2nd order effects like reputation damage, lost income etc can be adressed by lawyers.

Yes mental distress can be severe to some people. Or (for example) cause a group of people to harass someone.

But a) such harassment is a separate act from posting a deepfake. And

b) If deepfake postings happen often enough, it kinda takes the sting out. Few people today would be deeply disturbed if someone posts their head shopped onto a pornstar's body. That's easy enough that anyone can do it, it's been done many times, so the newsworthiness / shock value is gone.

If you know the victim in person, you'd probably be able to tell whether some video is fake or not. Or ask that person. If victim is a random stranger & deepfake disappears into the 'internet background noise', who cares.

Either way, when creating a deepfake is easy enough, it simultaneously grants a victim plausible deniability.

So ehm... "horrifying" is too strong a word imho. Try "annoying", "bad taste" or "disrespectful". Punishment (if any) should fit that.

THAT is horrifying. It's horrifying both by the base actions at play and, far moreso IMO, that because someone in the tech space thought it was monetizable enough to warrant investment, it will happen. It did happen. That genie is now out if it's box and when confronted with the ethical dilemmas it seems most people in the engineering space are content to shrug their shoulders and go "shit happens."

That's horrifying. It's horrifying that all a technology has to do is demonstrate it's ability to make money, no matter how socially or culturally corrosive it might be, no matter how much harm it might do to individuals or even entire professions, no matter how unethical and just straight up creepy it is, doesn't matter. If it will make someone rich, it will be done, and it will be done with the best hardware and expertise money can buy, and that social order will be corroded, that culture will be corroded, those individuals will be harmed, that profession will be obliterated, and those checks will be cashed.

THAT'S horrifying.

The #2 cause of death for people in the US aged 10-14 is suicide. Suicide is #3 for ages 15-24.

> b) If deepfake postings happen often enough, it kinda takes the sting out.

Ask any child subject to chronic bullying whether the repeated abuse "takes the sting out".

At some point soon every teen will know how to generate pictures of anything they can imagine. Teach impacted teen girls how to make deepfakes too and that it is not real. The world is not ready for this, but it is happening and can not be stopped so education is all we have left.

It is only shocking right now because it is new and seems like magic.

We should be educating teens on body positivity, and that anyone that shares a deepfake of them is announcing of how big of a child they are, and why they should not be taken seriously. It will just become an eye roll once everyone can do it and understands how easy it is. Background noise.

terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line?

imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?

At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.

I am not saying that is okay, but I am saying it is impossible to stop without shutting down the internet.

If anything deepfake tech reduces the demand to abuse real humans to obtain some types of pictures, which might reduce harm, even if in gross way.

Also you can find copyright movies too just as easily no matter how many billions have been spent trying to stop it.

AI is taking off like a rocket. Soon it’s going to be as easy as asking chatGPT to generate the photos/videos/whatever you want - and while chatGPT may be censored, there’s almost certainly going to be open source models that are not censored

What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?

Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning.

I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.

If it's about comparing generations, it's quite likely that your grandparents (or at least their parents) won't be here to suffer the consequences of modern society (I mean climate change and biodiversity loss), but you will.

So yeah, I'm not convinced that we have it better than a few generations ago.

Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.

These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"

Countries have tried this experiment like China and North Korea, and it seems to only serve those in power, not the public.

Let us say we did ban all open source decentralized technology in the US, and all websites that to not follow government rules, and mandate all digital communication in the US happen via government approved apps and devices.

How long do you think it will take before a US president with authoritarian ambitions decides to take advantage of this single point of failure?

Free anonymous and private communication is mandatory for a functional democracy. It has some bad side effects, but the side effects of not having it are most assuredly worse.

Out of curiosity: can you mention one democracy where people generally have "free anonymous and private communication"?

I can't think of a single democracy where journalists, dissidents, and whistle blowers are not able to communicate and coordinate or even transact with others anonymously and privately when needed, thanks to decentralized tools like encrypted email, matrix, tor, bitcoin, etc.

In fact sometimes when a given state oversteps, like Texas on abortion, it is -critical- that people can communicate anonymously to preserve basic human rights that other states still recognize.

If that was ever restricted we are in for a world of pain.

Most people and companies don't use encryption to protect themselves against governments, even though it would not be that hard (one could use ProtonMail for company e-mails, Signal/Threema/Matrix for messaging, etc, but instead everybody seems to be using GMail and Slack/Teams). Being anonymous is actually hard.

So yeah, the only thing that "somehow" prevents the US government from spying on everybody is law, because they clearly have had the capability for a long time. And given the occasional whistleblower, it feels like it's not actually enough.

Actually I would even wonder if access to fairly-anonymous and private communication is not more important in non-democracies. Because in a functioning democracy you should be able to discuss matters more freely (at least to some extent).

Anyway that's probably off topic, and I didn't mean to say that governments should ban encryption (encryption does exist, bad actors will always be able to use it even if you ban it). But that does not mean that governments should not interfere with technology in general, I would say.

Yeah not sure, maybe I'm completely off topic here.

1. allow encrypted anonymous communication and the ability to use that to defend democracy

2. disallow encrypted anonymous communication and the ability to get away with sharing deepfake nudes

Both of these are bad, but one is clearly much worse than the other.

Anyone calling for steps as drastic as internet censorship to stop one comparatively minor issue are clearly unaware of the consequences of that road.

This is especially relevant given that every decentralized system is actually an illusion, a merely temporary state of transition from one centralized regime to the next.

Well, not everybody has the resources to make everything. Pretty sure ChatGPT would not have been written (and trained) by a student in their room.

The open source community is actually going much faster with LLMs than the corpos, as usual. They can share information and collaborate while the corpos are too focused on competing.

It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.