"must" is probably the wrong word, but if they don't develop it, someone else will. At most, it would be delayed, and I'm not sure there's much difference in that.

Can we agree that “someone would have done this anyway” is a poor justification for one’s actions? That’s what you say when you 1) recognize that you did a bad thing, but 2) don’t want to be blamed for it.

Can we agree that it's fundamentally not a bad thing to develop a technology? If you were a legislator, how would you like to criminalize the development of technology?

Many things can be unethical/immoral without being illegal, and of course there are some technologies that are fundamentally bad.

Tech is such a malleable thing. Let's say that this repo doesn't exist, but repos exist for doing facial tracking on webcam feeds and pictures (for facial recognition), and others for modifying a 3d model according to a facial model output by facial trackers (for 3d avatars).

Let's say that the tech around this DeepFaceLive repo develops to the point that the repo can become just a 10-line file of glue that makes a 3d model that's just a grid plane with a picture on it that moves according to a webcam feed. Such a repo is redundant and anyone that wants this tech can just apply the repos mentioned in the previous paragraph to do the equivalent.

Are those repos now bad? If not, it seems the ethical question can be resolved by some restructuring of code so bad uses are not made so obvious, while being almost as easy.

What technologies are fundamentally bad? I don't know of any outside of perhaps torture devices.

Every single "weapon of mass destruction" is fundamentally bad technology, just for one ultra-easy answer…

Not sure. It may be that the world has become more peaceful since their creation, partly due to their deterrence effect.

The possibility that nuclear deterrence has helped the world to be more peaceful in comparison to the past means that it's not such an easy answer. Hasn't even been a century since nuclear weapons were invented though, so no clear statistics.

That's simply not accurate?

Nuclear weapons and the advent of mutually assured destruction has ushered in a world where many, many, many fewer people die of violent conflict.

Technologies that cause destruction are not congruent with destruction. They change the game such that destruction may actually be prevented by their existence. They are tools, nothing more.

It's possible they have permanently eradicated world war. If that's true, they are not fundamentally bad but in fact really quite great. Only time will tell.

There's surely at least some that you don't know of because their developer made the decision to not release.

No, we can't. Developing "tech" is an action bound by morals like any other action. Cigarette production has tech in it. They don't just grow on trees. It was developed, and keeps being developed, by people well aware of the effects of smoking.

You're opening another can of worms with that example. An adult, informed smoker that takes care to not cause second-hand smoking and that doesn't have any dependents, causes harm only to themselves. If they think that the reduction of stress or whatever else they gain at the tradeoff of their health is worth it, that's their choice. That's part of their bodily autonomy. As such, the tech is still not inherently bad.

You're conveniently dismissing a few concerns and from that drawing the conclusion it isn't inherently bad?

> Can we agree that it's fundamentally not a bad thing to develop a technology?

I do not agree with this assertion.

It’s simply that it’s a kids/teen level argument. “But they did it too!”

Not a lot of thought went into it, is what it usually tells me.

The argument wasn't "Doing something bad is fine because others are", it's "You're trying to stop a bad thing happening by stopping some people upstream from doing something that's perfectly fine. That's going to be ineffective."

But you're right about me not putting a lot of thought. I didn't think this would need more arguing than that.

I still believe that tech is fundamentally neutral, and as such so is its development. Even if a developer intends to give it bad use, the use and development are separate questions.

There are others - not you - who will justify their own actions thusly: the technology existed, its better that the good guys (me - of course) should use it before the bad guys do.

In the space between both equivocations (yours and the other guy's), lies the potential for moral abuse.

That's an even more faulty argument, though, because almost everyone considers themselves one of "the good guys".

Indeed, I think terming yourself in good vs bad as an we vs them terminology makes several biases go undiscovered that if you truly go for objectivity despite your sense of moral leaning is telling you.

What even is a good guy use for Deep fakes besides some memes that get old fast?

This "kid level" argument fooled even Albert Einstein, Roosevelt, and many others. into developing atomic bomb. "If we don't build it Nazis will"

Using it was achieved by different kid level argument "It will prevent even more death and suffering"

The difference is that a lot more thought went into it. Not only that, the presidents domain of expertise and distinction/authority is about these issues. You way down your options, you navigate a complex issue with many factors involved. To prevent more death and suffering was indeed what was one of the most contributing factors of Roosevelts decisions (also to use it, as a direct assault on Japan would have cost a factor more in lives and suffering for both Japan the US)

The difference is of course a well thought out weighing of options, to feel into the issue and make your choice. Taking my immediate surroundings, a lot of people don’t engage in that level of philosophy. Knee jerk reactions are still very common in our species.

Absolutely correct. But that fact doesn't make these arguments acceptable.

I actually don't agree with this. Assume that the following premises are true:

* action x is legal, but results in a worse world for everyone. * doing action x benefits the first 1000 people who do it. * much more than 1000 are capable of doing x, including me. * if I do it now, I will be one of the first 1000.

If these are true, I think one should do action x. I believe one would be completely justified in doing it.

Basically prisoner's dilemma.

I think the answer is inherent to the question of whether it's "justified", and the fact that you're being downvoted.

If Prisoner's Dilemma is played once, the optimal strategy is to defect. If it's repeated an unknown number of times, the optimal strategy is tit-for-tat.

Now, if you're an identifiable actor in a pool of tit-for-tat-players, and you have a history of defecting first, you're going to face opponents who normally play tit-for-tat start to defect against you first, too. In the end, you will end up as one of the worst peformers in the player pool.

So when you're asking if it is "justifiable" to defect in a game like this, it is the same as asking "will I be treated worse if I defect in this game". The downvote confirms this, I think. It means that in this population pool, bad behavior is punished (or people pretend it is)

Now, if defecting can either be done secretly or if you're in a popluation pool where everyone defects anyway, then always-defect is probably the nash equillibrium. In that case, defecting will be seen as justifiable.

> Basically prisoner's dilemma

> (...)In that case, defecting will be seen as justifiable.

Exactly! My argument basically boiled down to this. In certain situations, defecting may be the way to go. I would not blame the defector for doing what is in their best interest.

And “someone would have done this anyway” could very well be what's turning a situation into a prisoner's dilemma, so it could actually be a valid justification in some cases.

But yeah, it could depend on what is meant by "justification" exactly.

This assumes that there is no morality. That there is no merit in not doing something evil even if others will do it in your place.

Hmmm. Not necessarily. I mean if what you did does not change the outcome in any way, then is what you did even evil? If you have no marginal impact, can you ever do good or evil? I am not sure actually.

I realize that this could also apply to lower rank soldiers involved in some war crimes, etc. so I guess it is not that clear cut. But would you blame a random soldier of lowest rank for carrying out terrible orders and not risking himself(or his family)? Well I don't know. I guess I would not.

Of course, the stakes are rarely that high, but the harm done is also rarely that high also.

I don't know that I'd blame them, but I would consider it an act of evil

This is a bit of a tangent, but that's something I never liked about referring to evil things as "inhumane". It seems to me that hurting others in order to help friends, family, or self is very human. In that case, morality is choosing to act against human nature by doing the right thing at the expense of self or loved ones.

For an atheist, moral absolutes are absurd (though still widely held). There are no more an absolute case for saying it's evil for a human child to play before a mouse before killing it, than it is for a kitten to do the same.

Rather, one could argue, morality is a set of emotions, norms, beliefs and rules we have gained through culture and evolution in response to certain behavior patterns. And these can be seen as having a game-theoretical foundation. The rules that assigns the label Evil to someone tends to select people that we (or our ingroup) are in an existential conflict with, at a level where it's either us or them.

For instance, we can have rules that define some patterns of behavior as Evil, and have different set of moral rules for treating people depending on whether they are classified as Evil or not.

> I would not blame the defector for doing what is in their best interest.

I absolutely would. The defector is willingly harming unconsenting others. That the harm would have been caused by someone else anyway doesn't suddenly make it OK.

Not only that, but if some people have ethical and moral considerations about the consequences of x, then those people have an ethical reason to actually do the thing, since the alternative is that the people doing it are those with no ethical or moral consideration.

If a piece of technology is going to be 90% bad 10% good and you can push it to 80% bad then it is completely justifiable to go for it.

I could not disagree more.

That logic is how every atrocity in human history was enabled:

- If I catch this escaped slave, I can get the bounty instead of someone else!

- One of us Auschwitz guards will shoot that escaping prisoner; why shouldn’t I get rewarded for it?

So on and so forth.

It's a self-contradictory and bad-faith argument - it doesn't give another person the opportunity to say no, let alone to not do it.

You’re speaking to the wrong audience. “Someone else would do it anyway” is how everyone here justifies doing unethical shit at FAANG and refusing to earn less than $300k/yr base because “that’s just the industry”. I have no respect for anyone that says this shit with a straight face.

It should also be said that anyone that makes this argument immediately loses their right to complain when governments need to step in and regulate.

You're commenting even after someone has already provided a legitimate good use of this tech, and are the third person to write as if it were established that it's unethical to develop this tech. I never said that it's fine to do unethical things because others will do it anyway, and I wish you wouldn't imply that I did. From the get-go, it was an open discussion whether the development was good/bad/neutral, and my original comment was to clarify that jakelazaroff's comment probably misinterpreted what they quoted from the use of "must".

My position is that development is not unethical. I'm not trying to justify this position because I don't think it needs to be. When I said "others will do it anyway", it's not a justification and my original comment wasn't even about my position on this. My comment was referring to the usefulness of stopping the implicit always-ethically-neutral development of a tech in order to stop the potential misuse of it. I'm saying that even if they stopped the development of this one repo, or a few, or all others implementing this tech, it'd be ineffective. This repo is just one bit of the tech. Most of the tech is in its dependencies. If someone wants it and has nefarious uses in mind, they don't really really need this repo for that.

> My position is that development is not unethical.

I think this is where we disagree at heart. I don't think that development is an ethically neutral act, any more than any other human activity is ethically neutral.

I see this as part of the expansion of knowledge. People conduct research to get a better understanding of the universe. Tech development is about our understanding of what we can do and how. People conducting studies and research can be unethical in the process, same as tech development, but I can't think of knowledge that would be unethical to expand in if the process is ethical. What understanding of the universe would be wrong to obtain if ethical means were available?

If there's none, why is expanding our knowledge of what we can do and how any different?

I draw a distinction that you don't here. Development is not about the expansion of knowledge, it's about producing things. Science is about the expansion of knowledge.

As a society, we very often conflate the two, but they're really very different things.

They're very intertwined. A paperclip factory produces things; they don't do tech development (shouldn't do much at least). Tech development is about pushing the edge. Science, obtaining more knowledge about the universe we have to work with, helps, but it's also about finding new applications for what we've already known.

Tech development doesn't necessitate the production of things, even intangible ones. Technology development in the form of new optimization techniques in existing software for e.g. rendering 3d models or searching data faster or whatever is not producing things, but it's still tech development.

Technology is the practical application of knowledge about the world. Science is the acquisition of such knowledge. There certainly can be an overlap in the Venn diagram here, but the distinction remains valid and important.

In any case, I was not saying that your perspective is incorrect. I was just saying that my worldview seems rather different from yours.

If I can become rich instead of someone else, and the damage will be done either way, it seems perfectly rational if I can avoid any actual legal punishment (I don’t care about people on Twitter being angry with me, Twitter isn’t real life) to push the big red button and reap the rewards.

Once I’m rich and powerful I can afford to be ethical, isn’t that the way of billionaires anyway?

If a person isn't ethical in the first place (as proven by becoming wealthy through unethical means), they certainly aren't going to become ethical after they're rich.

Right. Tell that to Einstein, who just before Hiroshima realized it was not such a good argument.

That is a nice piece of history.

Ethics is important. And one consequence is: I may die but at least it was not because I made it possible.

Yeah, theory of relativity and tech with use cases mostly in porn and fraud aren't really the same thing.

Your response doesn't make any sense. Care to elaborate?

Just search "Atrioc" in google and read any article. So far, this tech's main use is porn with people who do not want to do porn, who do not post adult content, who did not ever consent to their faces being used for adult content.

The second use of this technology is fraud. Go read the issue and the comments linked in the post.

And you should not even need this much info. The simple fact that they mention "mrdeepfake" as one of the 4 communication groups, the second after their Discord server tells you how good this tech does to the world.

As GeorgeMD pointed out, there's some required reading.

What George is saying makes total sense. Why I had commented on your post was I couldn't understand your reasoning that somehow relativity and the 2 bombs couldn't have happened without each other I found that quite bizarre..

> that somehow relativity and the 2 bombs couldn't have happened without each other I found that quite bizarre..

Sounds about right, because that's literally not what I said.

It’s not relativity. Einstein pushed first for the american atom bomb because “the germans will make it anyway”. Then he realised that it was way too harmful and recanted.

Nothing to do with relativity, just with “if I don’t do it others will”.

What it boils down to is: If someone else will just do it, then wait and let them do it. If it has morally objectionable consequences, either you care about that and do not involve yourself, or you don't and you go ahead. But then you cannot say you didn't get your hands dirty.

While Einstein regretted working on nuclear weapons, the work they did there is equally responsible for the many lives saved by nuclear power, which he also appreciated. Also, it was the first 20% of the work towards nuclear fusion.

Technology is not evil or good.

Even this technology could be great. Jacky Chan, for example, is not going to make any more movies. But I want more of those movies. This very technology would allow that.

Does Jacky Chan have any saying in this? Or is anyone on the internet free to use his face however they like?

Yea, tech is not evil or good, it's all about how people use it and so far, deep fake tech has only been used for evil. From deep fake porn to fraud. The only not evil thing about this are the memes, but we can live without deep fake memes just fine.

> he work they did there is equally responsible for the many lives saved by nuclear power

This is still on-going. There are thousands of nuclear heads on standby. His technology might end human civilization or even the human race as we know it.

> Technology is not evil or good.

Of course, it's not. Technology is real. Evil and good are imaginary human-brain constructs.

The Nuclear bomb in itself was a lifesaver in the way that the US nor the USSR ever dared to attack each other because of the potential consequences.

Without Nuclear bombs a "hot" 3rd world-war would have probably happened.

Also no country who has nuclear weapons was ever attacked.

> Technology is not evil or good.

But a given technology may be much more suited to doing evil than doing good.

> Jacky Chan, for example, is not going to make any more movies. But I want more of those movies. This very technology would allow that.

Which is something that I would consider ethically wrong unless Jackie Chan consented to it.

From the main page: “Communication groups [...] • mrdeepfakes - the biggest NSFW English deepfake community”, which strains my assumption of good faith debate regarding their ethics.

Nothing about producing or consuming deepfakes suggests bad faith.

You might not like the art, but deepfakes are not harmful.

This is a shallow take, no matter which way you look at it. "Guns don't kill people."

“NSFW deepfakes” literally means “porn with real people’s faces inserted.”

Maybe that’s an art form, but it’s also clearly potentially harmful in the same way as revenge porn, teenage nude selfies, and other cases where regular people and porn intersect.

99% agree. The 1% is for the potential for deepfakes to solve blackmail. If producing deepfakes becomes trivial, any blackmail threat can be dismissed with “Whatever, I’ll just say it’s a deepfake”.

This is not how humans work. False statements about someone damage that person, even once everybody knows they're false.

But with this argument you could blackmail someone whether or not you have some evidence.

You indeed can. Any of your coworkers can report you to HR right now for allegedly saying to them in private that you find underaged boys attractive. Wouldn't that at least be quite stressful to you?

Personally, I've been on the fence about doing sex work. On the one hand, I'm nervous about people identifying me based on my face or identifying marks, but lately I feel like I shouldn't care, I should be open about it and that way I can't be blackmailed.

This sounds like something literally 90% of humanity does in their imagination at some point in their lives. It's inevitable that someone is going to do it in a video when the tech becomes good enough.

I'd say it's not the thing itself that's a problem, it's publishing whatever you create as if it's real. There are many things in life that are potentially harmful, but are part of everyday life since nobody actually harms anyone with them.

You do see the difference between a thought, shared with no one and visible to no one, and an image on a computer, which can be uploaded to the internet, right? This isn't about stopping sexual desire, it's about not distributing images of someone's naked body without their permission. That it's not a guaranteed accurate representation is sort of beside the point. It's good enough if you've never seen them naked before. Some people have gone on record saying they don't like it when it happens to them. Why, do you think?

> You do see the difference between a thought, shared with no one and visible to no one, and an image on a computer, which can be uploaded to the internet, right?

Why do you assume it’s shared with no-one? The representations change, but I can guarantee you that Bianna the Beautiful had several suggestive clay figures created of her without her permission.

Not to say that that was necessarily great, Bianna would probably have been upset too. But trying to fight the method of representation instead of the sharing seems like a fools errant to me.

Why is it clearly potentially harmful? It is not clear to me at all, as photoshop already exists.

How is someone harmed by someone else producing video that features people who look like them? The whole "seeing is believing" thing hasn't been true for ages, every movie is half CG these days.

I would perhaps buy the premise if deepfakes were sprung on the scene in the 70s or 80s, but the 90s and 2000s have inundated so many with completely fantastical high res CG imagery that nobody thinks videos are proof of anything anymore.

Wouldn't there be a dramatically different level of effort to photoshop every frame in a video convincingly than to create a deep fake of the video?

Do you agree that most people wouldn't like to find a nude video of themselves online?

Now why does it make a difference if the video is real or not? It's still a video of a nude body with your face on it.

I find this whole deepfake-porn trend to be incredibly disgusting. And it just saddens me that heterosexual males always immediately have to exploit such things for their own hornyness. I guess this is also the reason why this topic is not being criticized as much, because the people who develop and consume these videos can't comprehend how disrespectful it all is towards the people (women) who are being deepfaked AND especially the results being shared online, in most cases publicly.

I think that most people would not care if a video is posted online of them if the video is in fact fake.

Would you care if someone posted a nude image of someone with your face photoshopped on it? If so, why?

This recently happened to a large number of women that are Twitch streamers and they are all very upset. You are free to seek out and read their explanations as to why, I'll include a few tweets.

https://twitter.com/qtcinderella/status/1620264657926885380?...

https://twitter.com/mayahiga/status/1620586546083803136?t=c7...

> Would you care

Yes.

> why?

Because it's insanely creepy for someone to photoshop my face onto a naked body without asking me and then post said picture online?!? Especially if it's done so well that people don't notice that it's not actually me.

If your rebuttal is that "not everyone cares if deepfaked photos of themselves are being published online" then the answer is pretty simple: As long as you don't know if someone minds it, don't fucking do it.

It's really alienating to me when I think about the fact that we're discussing if it's okay to upload deepfaked nudes of someone without their consent.

The only reason I can come up with is that some people have probably consumed lots of deepfakes and now don't want admit that it's maybe a bit creepy and wrong.

I just don't understand why anyone would care?

You don't need consent from anyone depicted in visual art to make visual art. It literally has nothing to do with them.

I see no moral, ethical, or consent issue with deepfakes whatsoever. People are allowed to make whatever sort of CGI they can imagine, and, most critically: they should be.

> You don't need consent from anyone depicted in visual art to make visual art

We're talking about (mr)deepfakes. Which is visual art, yes. Nude visual art.

That's kinda the point. The nudity is what makes it immediately not okay.

If you just deepfake someones face onto Jason Stathams body, they will probably enjoy and laugh about it.

If you deepfake their face onto someone who's getting roughly f*cked in the ass, they might not like it that much.

But then again, not just nudity is wrong. Deepfaking someones face into a video to hurt them is wrong as well.

I guess what I'm trying to say is that I really think that AI Ethics should be pushed harder and more seriously.

If someone deepfakes their favorite actresses head into a porn video, I still think that it's wrong. What I would appreciate is if I can at least agree with this person, that what they're doing is not 100% okay.

> You don't need consent from anyone depicted in visual art to make visual art. It literally has nothing to do with them.

Yes you do. https://en.wikipedia.org/wiki/Personality_rights

Have you asked the people this happened to or are you just making assumptions? It’s important to critically examine our own biases in things like this, particularly if we have no idea why something evokes a strong emotional reaction.

> I think that most people would not care if a video is posted online of them if the video is in fact fake.

I think most people would care a lot. After all, it doesn't matter if it's fake or not, if matters if anyone you care about thinks it's real. And some almost certainly will.

We've not had the test case, but in places where nonconsensual or "revenge" porn is criminalized, deepfakes may well turn out to be illegal. They're certainly distressing for the victim.

It's already illegal for a person to pretend to be another real person without their consent.

This needs an asterisk (and I'll include one of my own: IANAL). In the United States, impersonation is regulated by the states and not consistent across them, but in general this is only true when using impersonation to commit fraud, defamation, or another act that itself is likely a crime already. Many cases would be considered parody or satire, even if no one finds them funny.

This may be the case in some places, and fraud is illegal in general, but should it be illegal to make parodies of politicians using paid actors? I think most people would say no.

Then why should it be illegal to do the same with an AI?

> strains my assumption of good faith debate regarding their ethics.

Let's flip the argument upside down. Let's imagine you want to produce porn content, but don't want to be recognized. This technology allows that.

Side note about that: people doing amateur porn where their face doesn't show at all still get recognized because of particular marks on their skins. So this might help this very specific case, but maybe not as much as one would think.

It's trivial to use Stable Diffusion img2img with low noise to change the marks on your skin.

this fucking guy

I've already seen porn where someone's face is blurred.

And whose face are you going to put in that porn with this?

Emma Watson, probably

The readme for the repo has examples of AI generated faces. Think this person does not exist, but better.

If someone makes a deepfake on their own computer, watches it, and doesn't share it with anybody, I don't see how that's markedly different (morally) from just imagining the same thing. Some people have a very strong visual imagination and others don't have it at all, it's only fair if they can use a technological substitute.

Also some entertainment works by artificially instilling a desire which cannot be fulfilled. If people can use deepfakes and masturbation to defuse that desire it might be a moral positive for them.

https://en.wikipedia.org/wiki/Aphantasia is a real thing and affects about 4% of people.

mrdeepfakes.com is a site where deepfakes are being shared publicly, so your comment doesn't have much to do with the comment you actually replied to.

I think it's a fundamentally different thing, morally speaking.

Imagination is fleeting, ephemeral, jumbled, and usually linked to a specific state of mind that passes once the person has either grown bored of the imagination of achieved whatever satisfaction they wanted from it.

A deep fake image or video is persistent on your device and may leak some day, but worse is that feels real in a way mere imagination never can and it's something a person could go back to over and over, feeding into and enhancing their obsession with the non-consenting person.

I think it's a bad thing in ways that imagination and porn are not.

What if I photoshop my crush's face onto a naked body? Wouldn't that be similar to having a personal deepfake?

I agree with the person you're replying to, you can't stop people doing these things in the privacy of their own home and these deepfakes are functionally similar to other photo / video editing techniques.

_Distributing_ those photoshops / deepfakes is another thing entirely though, and one I am fully against.

The question wasn't whether people can be stopped from doing it, because for the most part they can't be, but whether it's moral. Or at least different in a moral sense from merely imagining someone else doing those actions.

And yes, photoshopping your crush's face onto a naked body is just as unethical and immoral, even if done in the privacy of your own home, for the same reasons I mentioned above.

Compared to deep fakes, however, it has a much higher effort, lower reward, and less realism, which acts as its own inhibitor. Deep fake generators are becoming trivial to use, and not just for one or two pictures but for as many videos as you'd want. That's going to result in a very different driver of obsession.

Morality evolves with technology. At some point in the future even killing someone migh not be very immoral if technology reaches the point where we can back up and recover from back up.

Maybe, but what’s the relevance? We can’t predict that, nor can we adjust today’s morality to match what technology might enable in future.

Perhaps murder will one day be not very immoral, but that doesn’t mean we should treat it any differently today.

Right now, creating fake naked pictures and videos of real and non-consenting people is immoral. In some jurisdictions it’s illegal. It’s also probably harmful, by taking obsessions further than mere imagination would allow.

> photoshopping your crush's face onto a naked body is just as unethical and immoral, even if done in the privacy of your own home, for the same reasons I mentioned above.

The reasons you mentioned above were "it feels more real than imagination" and "a person could go back to it over and over, enhancing their obsession with the non-consenting person". These are not moral bads that require a "no lusting after people, even in your heart" style rule. Being obsessed with people starts in high school and can persist for decades. They never find out, and it never hurts them.

I'm sure many people would like to take away others' ability to picture them naked, but fortunately they didn't have the ability to enforce that by starting a moral crusade against a Github repo. It's one of life's innocent little pleasures.

I never said this was about a "no lusting after people, even in your heart" rule, nor do I believe it should be. If someone lusts after someone else in their head and never does anything to harm the other person there's obviously nothing wrong with that.

It's immoral because it's creating something persistent that could harm someone without their consent, amongst other things.

I also question your assertion that obsessions of this type are always harmless, because it's obvious that many are not and escalate into outright stalking, confrontations, or worse, and anything that makes that sort of obsession more likely or helps to enhance it where it exists increases the chances of that happening.

> What if I photoshop my crush's face onto a naked body? Wouldn't that be similar to having a personal deepfake?

Yes, and it's already widely considered to be really fucking weird behavior

My childhood was typical. Summers in Rangoon, luge lessons. In the spring we'd make meat helmets.

Of course it took years for me to perfect my artistic skill in the area of erotic fingernail sculptures and by that time anyone could afford a Photoshop license.

Fully agree with ya there, but is it _immoral_? I would say no

TBH I got curious. Clicked on one celeb, oh man, I've seen 90's Photoshop cut and paste the face onto a different body better than those "deepfakes"...

We do in fact have a gun to our heads. I don't want to ruin your life but those who know know and it starts with R and B. Look it up at your own risk.

Come on, everyone knows that ruining your life starts with metal. R&B, while annoying, is quite safe.

>Technology isn’t inevitable. It’s a choice we make.

See and that's the problem, you can just talk about "you" not "we", because someone else will do it. Technology, and with it information cannot be stopped.

"I didn't murder that man, your honor, he would have died eventually anyway."

Incredible stupid comment...just to extend your logic the thing you type right now is "enabling" that "murdering"...so does the invention of the wheel enabling fast wars...or bring you fast into a hospital per ambulance.

As you can't enforce stopping development of it, it is better to also develop it to understand it better and create countermeasures like ml which detects deepfakes.

This also leads potentially to more or better image/video signing Technologie.

And yes just because someone develops this in the open doesn't mean that someone else is not developing it in parallel but hidden.

There is ZERO credibility that this development will be used to "create a countermeasure", especially mrdeepfakes is a key communication channel there.

I'm not talking special to mrdeepfake.

My argument still makes a point that you can't just forbid it. Someone else will do it anyway

Of course you can, even if somebody else does it. Case in point: any penal code.

How can you force stopping development of software?

One of the more egregious examples: https://www.wired.com/2011/03/aleynikov-sentencing/

> No, it mustn’t.

Let's imagine you have a horrific face malformation, or went through a terrible accident that left you disfigured. Why not allow you to pass for an ordinary person, at least in video calls, so you can have a somewhat normal life?

It took me two minutes to come up with a legitimate use for this technology. I imagine there are many more only a couple minutes away.

I took the OP's comment as meaning "it's not necessary for this to happen" rather than "it's necessary for this not to happen", a nuance which can be confusing, especially for non-native English speakers. The comment in the article suggests there is no choice but for this technology to be developed, and I think the OP is disagreeing with that assessment and saying "there is a choice".

If I've understood that right, both your comments can agree. There are strong arguments to say we _should_ develop this technology, but there also many good counter-arguments.

> there also many good counter-arguments.

Someone will develop it, and someone will use it for nefarious purposes - as one terrorist organization spokesperson once said, "We only need to get lucky once. You need to get lucky every time". If it is developed like this, in the open, at least we have a better chance to understand its capabilities and limitations, and to be better equipped to build countermeasures.

This is exactly the Prisoner's Dilemma, and also shows up in arguments why we need to build insane numbers of nuclear weapons. The logic rests on a faulty assumption that there is a binary choice - either you build it this way, or somebody else builds it this way. It doesn't consider any other possible paths that could lead to better outcomes.

There is an economic incentive to build it this way - for all the legal, and illegal applications it may be used. Movies can make stunts look more like the actors they portray more easily (this has existed for decades now), you can pay Tom Cruise to use his likeness on a publicity campaign without having to pay for his time. You can change your face, or your body, so you are not recognized.

It's a virtual certainty that a technology that can have a commercial application will be developed. I can't imagine a plausible path departing from this present where it isn't.

Yes but in the end it’s all about choice. It’s not some creation of chaotic coincidences and circumstances. It’s not some accidental discovery. It’s an active choice by whoever decides to make it for whoever purpose.

The argument isn’t matter of should it be made, but that the only reason it’s being made is because of people actual choices to do so. It’s not randomly spawning Into existence of its own free will.

An extreme version of this might be:

> I saw Bob about to stab Alice, so I beat Bob to it and shot her in the head. I did nothing wrong since the outcome would have been the same regardless of my actions.

On the bright side, you probably saved Bob's life.

Yeah, that’s correct. I wanted to match the original comment’s phrasing and thought it was clear from context, but I probably should have just used a different word to avoid ambiguity.

It is much more useful for privacy concious people not wanting to give Zoom more data for their face detection database. All in all it's good for humanity when technology can counteract other evil tech that will spy on you.

you could use avatars / 3D models (possibly a few realistic ones) for those, no need for deepfake technology from a single photo.

True, but that'd be obvious. For a lot of use cases the point would be to do it without being detected.

For technology not to be developed, everyone with means to develop would need to choose not to. It's not too far from "inevitable"

Potential technology is an infinite set. A gigabyte contains 2^800,000,000 possibilities; a number billions of digits long. Many are junk, but many are not, and even out of that we choose a small subset that we think is worth pursuing.

Saying “everyone with means to develop would need to choose not to” is saying like “everyone would need to choose not to write this book”. It’s not inevitable.

The problem is that there is a big confusion about building the technology and using it in some specific, wrong way. Surely anyone here will agree that building the technology itself should not be forbidden. You can do it for some valid reason, or simply for fun. It’s not for anyone to judge or stop you from doing it. Those who use it for some nefarious reason must then answer for that. Guns are the obvious comparison (although maybe not the best) as in they are needed for protection also. But another example is how explosive devices and technologies can be used in a variety of useful ways, such as mining, terraforming, etc. Using someone’s image without their consent is already a crime and applicable in many other non-DF scenarios.

What's the confusion? The allegation is that choosing to build this represents, at best, incredibly poor judgment. The GitHub issue and GP post are good faith appeals to kindest possible interpretations of this work. To use your own example, explosives can be useful, but if you discovered that your neighbor was building pipe bombs in his garage, you'd probably want them to stop. "I'm not gonna use them. I just think they're neat." is not a convincing argument for them to continue. To extend the analogy a bit further, this seems comparable to discovering a box full of pipe bombs with a big "free" sign in front of their house.

Plus... they're going to use it.

My analogies are horrible…

I understand the sentiment but I disagree. It's just another disruptive piece of technology that people will have to adjust to. If you can't trust a digital face, then nobody will and using this tech for scamming purposes will simply fizzle out after an initial "adjustment period". I know that sounds rough but society might react differently than you think, i.e.: with digital faces being useless for identifying people, and maybe even becoming creepy or unsettling because of the implicated fakeness, meeting people in real life (opening bank accounts, transactions in general, anything where trust is valuable) will have more value. Don't fight progress in the hopes you can one day become complacent.

It will fizzle out right after fraud involving the telephone or email died out.

I don't see a difference between any technology. So if we can frown upon some nation for developing nukes, we can frown upon a group people for developing this.

On the other hand somebody can say "if we don't, then others will". So regulate and ban the technology, then.

> So regulate and ban the technology, then.

When has that ever worked? What banned technology is conceived of but not developed because some government entity said not to?

I think the "if we don't, then others will" is part of the natural progression of technology. Whatever the next logical step of development is, that's where development efforts will flow. Some might not want to go there, but some will. Someone banning it will likely only fan the flames and drive more interest into the space - ala the "Streisand Effect".

When the US government (et al) labeled certain numbers "illegal" because they could be used to break DRM or certain encryption types, academia and hackers alike openly mocked the notion. T-shirts, stickers, and websites sprung up further spreading this "illegal" knowledge. People who had no idea about how a number could be so "dangerous" suddenly wanted to know. By telling people they can't know or do something will absolutely drive people toward that knowledge.

The hacker mentality often answers the question "why?" with "because I can". Saying you cannot only encourages more to jump in.

That a law is broken by people does not mean we don't need it.

I see banning things is useless in the long run, but at least some people will think about why it might be banned.

I was thinking about throwing a wrench to the machine while saying "ban it" to make it stutter, like breaking the chain of obedience in Milgram Experiment, or like the woman who walks up to Zimbardo and stops The Prison Experiment (see his TED talk).

Because, as a hacker and programmer, I believe that we have ethical obligations, and this "We're doing something amazing, we need no permission" stance in these communities genuinely worry me.

Technology is not only technology, it affects people's lives. Anything which can damage it beyond a certain point by exploiting human nature is in the same category for me.

Trying to protect low security by censorship is a repetition of history: https://www.zdnet.com/article/chilling-effect-lawsuits-threa...

It's like banning Iliad, because it describes troyan horse.

I think you're missing the point here. Because I don't say anything in the line of that.

What I say is, this is a double-use technology and dangerous beyond a certain point, so it might need to be regulated or banned at the end of the day.

In an ideal world, we shouldn't need this, but we don't live that ideal world.

For example, I can't independently develop an amateur rocket which can land to an area of my choosing by the means of actively directing itself, beyond a certain accuracy and precision. Because it'll be a homing missile. On the same essence I can say that this technology can be used to damage other people.

Or, I can't get some enriched uranium to build myself a small, teapot sized reactor to power my house during power outages.

Can we say that we're censoring research in this areas too, because they're low security things?

This is same with latest A.I. developments. However I'm a bit busy to open these cans today.

Nuclear technology is low security indeed, and it's a technical problem, and uranium isn't exactly abundant element. Untrusted data is a problem of stupidity in comparison. But stupidity causes problems with any technology. It "spoons can harm people" tier problem.

It's possible to buy anything given the right price when it's not regulated or banned due to double-use technology restrictions. I'm sure while expensive I would be able to get required equipment for the right money, from the usual suspects (i.e. I'm sure there'll be microcontroller boards for controlling reactors up to 200KW or servo kits for 12 fuel, 12 regulator rod configurations from adafruit for example).

> Untrusted data is a problem of stupidity in comparison.

In the past, wrong data showed itself because of a lack of coherency. With the advanced misinformation operations, it almost have became an alternative reality game. A.I. today allows us to generate convincing lies at the push of a button. I can fathom what kind of misinformation bubbles can be built with technology like that.

These technologies are attacking to lowest level instincts of humans, which ones we deem utterly reliable for thousands of years. They are the same level with the manipulative algorithms in my mind. I put these into dangerous and harmful category.

This is not a case of stupidity. This is plain old, and very dangerous kind of, manipulation.

Downplaying this is not wise.

I say frown away! We all know what this is used for and its tasteless at best. But I like to think in solutions, so in stead of trying to arms-race this tech into submission with ever cleverer deep-fake detectors (and thus be able to regulate), why not accept its existence and change ourselves. Not least of all because any digital tech is like a permanent mutation to the digital-dna of our global society.

> why not accept its existence and change ourselves.

That works too, like we accept the algorithms which tie people to screens and used to spread misinformation and manipulate the masses.

> We all know what this is used for and its tasteless at best.

Yeah, like interview fraud, misinformation campaigns and what not. These are tasteless, but not harmless.

"If we don't, then others will" is a bankrupt argument put forth by people who are trying to justify doing things they know are harmful. It would be more legitimate if it were "we won't, but others will".

This is what I was trying to point too, and why I added the "ban it, then" argument. Because there's a considerable amount of people who operate with the "if it's not illegal, then I can do this" mindset, and there's a big intersection of this two mindsets.

At least putting a ban on it will make people think, I hope.

Agreed.

I'm not sure how I feel about "ban it" -- I can argue that either way -- but I do think that arguments that banning something is pointless because people will do it anyway are misguided.

Regulations don't eliminate the effects of bad actors, but they do reduce the number and severity of them.

> So regulate and ban the technology, then.

Except for criminals and CIA.

> Don't fight progress in the hopes you can one day become complacent.

I'm not convinced this represents "progress". That aside, the goal of resisting it isn't to allow you to become complacent some day. The goal is to avoid being harmed.

Disruptive technology disrupts. Society and culture as well as markets.

How many generations did it take for society to adapt to the industrial revolution? What were the spasms which occurred during that period?

As for deepfakes, we know before we even begin that it breaks (utterly moots) social norms about identity, trust, reputation.

How many people are going to die while societies adapt?

Is that price acceptable for the benefit of more amusing viral videos on TikTok?

I disagree. We should develop this technology and make people aware of the implications and teach them to be vigilant.

Because being vigilant works so well now...

Knowledge isn't useful if you can't apply it in practice.

My work's IT has a habit of sending out phishing emails that closely match emails legitimate emails our employees and vendors send. We're supposed to be vigilant for looking at the Reply-To field, which is hidden by default on mobile devices. (As is the url things are going to.)

I will never not trigger those. Why? My brain can't tap the header field for every single email I read. And I can't ignore legitimate emails.

Every time on my phone, I'll click the link, then check the url in the browser. I can't get my brain to check before clicking the link. (Trust me, I'm neuro-divergent and I'd tried for years. It won't reprogram.)

I interpreted this as basically equivalent to "Someone will develop it somewhere."

> Technology isn’t inevitable. It’s a choice we make.

I think it is inevitable actually. The history of humanity seems to suggest so at least.

Agreed. Convincing folks who think that of anything else is quite difficult.

I’ll promote here one of my favorite reads, The Technological Society by Jacques Ellul. It feels very apropos here as we embark on an new smorgasbord of technologies that can do us harm and good. There will always be people who want to develop technologies even if they are harmful. We need not only ask ourselves “what is the benefit”, but “at what cost?”

> technology isnt inevitable

i disagree, society could stop inventing and humanity would likely continue to exist. However, that would would require the population to stop being interested in the development of technology which i think wont happen.

The point is, if this engineer doesnt write this code, someone else will. If a person can imagine a tool, someone will eventually want to make it a reality.

i think deepfakes and ai generation are going to change the world in a way that im a bit leery of, it feels safer to just stop here, but that isnt a possibility, even if deepfakes are made illegal, people will still create their own tools and have their own harddrives for storage.

maybe these things should be illegal or something, hard to say. But the tech will move forward either way imo.

We all have responsibilities as technologists to be ethical (to our own standards, at least), and to keep in mind the impact of the things we build. That applies regardless of whether other tech with a negative social impact exists, or for that matter whether it will exist. You can still be a good person, even if no one else is doing so, and you're under no pressure to be a bad person (in this context, anyway).