I fully agree with this sentiment - but what is the solution? I think most people do agree that such uses of this technology are horrible, but what can we really do about it? We can't put the cat back in the bag, and while efforts to censor the more popular services are commendable (if it's even possible to do so), that's not really going to stop bad actors from doing it

I don't pretend to have the answers, my point is more that if you want to be a part of the discussion of what a solution is, you need to have more than a shallow dismissal. We're still looking at the tip of the iceberg. Deepfake-enabled harassment, fraud and propaganda haven't seen extensive use yet. When and if they do, "you could do this with Photoshop" is going to sound increasingly out of touch.

This is a social problem enabled by technology, so the solution is necessarily a social one with technology to support it.

I don't know if this fits existing legal frameworks or if we'll have to put new laws on the books, but we should treat this as harassment. We should make it clear to younger generations that this isn't okay and why.

On the flip side, I imagine people will get more defensive about their biometric data. We'll probably start to regard photos and voice recordings as being very private. It probably won't be as socially acceptable to post them. It will probably be more normalized to use an avatar and synthesized voice on things like Zoom calls.

But this is all very speculative, so I won't be too surprised if things go very differently than I anticipate.

> It will probably be more normalized to use an avatar and synthesized voice on things like Zoom calls.

Which creates a subsequent issue of it being completely impossible to authenticate the person you're talking to. Which will inevitably lead to companies being completely cored out by exploiters who, on a long term, infiltrate networks and give instructions to subordinates 'as the boss' which lead to companies failing.

I don't really follow that chain of reasoning.

Sue everyone involved for defamation. No more immunity for hosting user-generated content because people abused the privilege. Problem solved.

The bad keeps happening because there is no financial incentive to change. Platforms are all too happy to look the other way because there's no risk to them. Change that.

As much as I find them distasteful, I'll agree that deepfakes should be outlawed as harassment the same day someone holds teenaged girls spreading rumors to the same standard.

What on earth are you on about. I just built some classifiers to run in industrial settings, why is it my job to do or say anything about Taylor Swift deepfakes.

> Because it's up to people like you to demonstrate to society that

Condescending as shit.

I'm sorry that I made you feel patronized.

If you just wanna make your classifiers and tend to your own knitting, then I have no problem with that. There are a lot of people in this community who vociferously advocate for AI, to the point that I view them as political activists for AI, and it's food for thought for them. But that's a mantle they've chosen to take on, it isn't mandatory.

It's interesting how 'any use of the tool' gets mixed up with 'responsible use of the tool' and that people who engage in responsible uses of the tool somehow feel addressed when it is very clear that you didn't have them in mind when you wrote that.

Yikes, please don't break the site guidelines like this, regardless of how bad another comment is or you feel it is. We have to ban accounts that post this aggressively, and I don't want to ban you.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Edit: Fortunately your other comments seem fine so this shouldn't be too hard to fix.

This is not how blame works. You have no idea who is reading your message.

I have no interest in blaming anyone. I'm talking about an attitude I see in this thread, which I believe is shortsighted.

Your post is warning about how other people will be held in judgement:

> People aren't dumb and they're going to notice that you aren't in their corner. And then they're going to start deciding what to do about AI without you.

I find it rather dubious. Who are these people who will remember what some random Hacker News commenters said? Why would they be listening to them in the first place? It seems like an empty threat.

It's not a threat, that's a pretty uncharitable reading. Society will realize that everything is not fine, and that the group of people who insist that it is are to be ignored. I don't imagine that will involve singling out individual HN users.

If this is a subject you want to discuss then I'm here to discuss it. If you want to dissect my language further, you're free to do so. But I'm not interested in arguing about semantics or tone and this is my final word on it.

Blame and personal responsibility are not necessarily connected. Blame is imposed from outside, personal responsibility is uniquely yours: you either accept responsibility for your actions or you do not and if you do then chances are that that will modify your behavior.

It seems like writing about how someone else should take responsibility is blaming by this definition, since it comes from the outside?

Or maybe a vague exhortation to do better doesn't count.

The harm here isn't AI, or if it is the harm is minimal. Media companies have routinely organised vicious smear campaigns since forever. If you spend time really watching politics, there are some grave injustices peddled by insensitive mobs. I expect we can date that sort of activity back to Roman times.

AI is going to make life pretty miserable for good looking women in the public eye. Maybe. I think we can probably social-out the interest as this becomes widespread, it isn't like this actually reflects badly on Taylor Swift. But that has always been a serious hazard of public life.

We've had this sort of thing happen before [0]; IMO this could be argued as harm reduction. I'm not sure the argument would be carried, but there is a case to be made. The important thing is to acknowledge this is easy to do and everyone agrees that nudes should be ignored.

[0] https://en.wikipedia.org/wiki/2014_celebrity_nude_photo_leak

> The harm here isn't AI, or if it is the harm is minimal.

> AI is going to make life pretty miserable for good looking women in the public eye.

Could you help me understand how you reconcile these two statements? Is the harm minimal, or will women in the public eye be miserable? Or is making women in the public eye miserable not something to take seriously - and if so, why?

I'd also be curious if you think the harm will stop there - could you foresee maybe, high schoolers using deepfake porn to bully each other, con artists using cloned voices to commit fraud, deepfake evidence being used to frame someone for a crime, etc? Is there a reason I shouldn't be concerned about these prospects?

Being in the public eye is already miserable. The public eye appears to be a close relation to the Eye of Sauron. These people smile a lot, but they don't look like they're having much fun.

The existence of AI will make life a bit worse for some of them. But life being better or worse doesn't really reach my standard of 'more harm' here. I suppose what I was thinking is a bit like if someone cuts in front of me in a queue. I might be worse off, certainly could feel annoyance, but I wouldn't say I'm harmed. I just don't like it. For harm to have happened I would actually have to be ... I dunno, harmed.

Ms. Swift already has to deal with literal stalkers. The baseline of what public people are expected to put up with is already high. I'm not sure this thing is really moving the bar.

> I'd also be curious if you think...

2 thirds of that (bullying, miscarriage of justice) are already major problems. The legal system is already a crapshoot; they do there best but it makes a lot of mistakes. I could see the net effect of AI being helpful in both cases; I'm more excited about neutral, AI judges than I am worried about evidence quality dropping off.

Con artists, we'll see what happens. The major protection I've relied on to date has been rollbacks and balances rather than preventing fraud up front. Social engineering is hard to stop.

You can feel concerned about whatever you like, of course.

To be frank this is nonsense. People in the public eye are miserable so it doesn't matter if they're more miserable? Making lide worse isn't harm? I don't even know what harm would mean, then.

I'm always bemused when people counter with the observation that something is already a problem. Okay? So then we should agree that more if it is also a problem, right? It's like saying the invention of the machine gun wasn't a factor in WW2, because we had already had guns and before that we had crossbows. There was a phase change with the machine gun and things were quite different.

Similarly, if people are able to produce deepfakes at an industrial scale, things will be very different.

> I don't even know what harm would mean, then.

Suffering some sort of detectable injury, for example. I'm hedging a bit because there could easily be something obvious I'm not seeing, but I don't really see how Taylor Swift of 2024 is worse off than the one of 2023 here. She hasn't suffered any physical harm. I don't think this harms her reputation. It isn't going to cause her financial prospects to drop. There is emotional harm, but that has always been a bit of a cop out taht people use when they just don't like something and I don't see why it has to cause emotional harm. There doesn't have to be any stigma around this, it looks like it is just going to be one of those things that happens.

Automated fake porn is obviously different and I can see why she might be upset. But it isn't so clear she - or anyone else - has actually been harmed in any serious way. I get upset by a bunch of things that are frankly more materially impactful on me than this is on her and it isn't appropriate to say they harm me. Tax springs to mind as a light example.

And speaking of WWII, we'll note that although war became much more horrific, it became so much more horrific that we seem to have stopped doing so much of it. So although objectively the machine gun makes fighting much worse, it actually led to less killing overall.

> There is emotional harm...

I agree. Things like this are potentially traumatic. This reduces to a form of cyber bullying. Cyber bullying has gotten people killed.

> [T]hat has always been a bit of a cop out taht people use when they just don't like something...

I disagree. Furthermore I would describe saying that there wasn't harm, and then admitting that there was but that it doesn't count, as a copout or equivocation.

> Tax springs to mind as a light example.

I shouldn't need to point out why harassment and taxation is an apples to oranges comparison.

> [A]lthough war became much more horrific, it became so much more horrific that we seem to have stopped doing so much of it...

We do a huge amount of war. I'll wager you're not aware of all the conflicts categorized as a "war"/"major war" in the list below (I wasn't when I came across it a few days ago).

https://en.m.wikipedia.org/wiki/List_of_ongoing_armed_confli...

I'm not even sure how to respond to the idea that war getting more horrific is a boon except to say that you should think on that again.

Bullying is targeted though. I doubt that this instance is someone trying to annoy Swift. She could theoretically not have noticed the episode at all and the faker would still be happy. I don't see why this has to make her feel like she is under some sort of attack. She isn't; most of the people looking at this stuff think she is a very respectable human being. Some fanatically think she is one of the best.

> I shouldn't need to point out why harassment and taxation is an apples to oranges comparison.

Sure. But you can see how (1) me being worse off and (2) me being upset about it are not sufficient for people to be persuaded that harm is being done to me. I mean, technically there is a point there but most people don't consider it appropriate to describe taxes as "harmful".

* War and Peace

There is a lot of war in the world, but if you look at the overall trends we're actually in a period of remarkable peace [0]. Absolute number of wars is way down, and death rates due to war will be way down (note that population growth is exponential so absolute numbers being a little higher actually means rates are dropping).

[0] https://ourworldindata.org/conflict-measures-how-do-research...

> ...deepfake evidence being used to frame someone for a crime, etc? Is there a reason I shouldn't be concerned about these prospects?

This is already happening, I'm reasonably certain, and until something of substance is done about it on a systemic, legal level, "deepfake" proliferation will lead to further systemic 'othering' of already-marginalized people at the expense of our own humanity.

Eventually, some of us (humans) won't be able to worry so much about engineering heaven-on-demand, as it were, when suddenly we find ourselves limited in ways we never could've imagined or ever hope to escape from.

People will continue to point and laugh by way of social media, but for how long and at what, or more importantly, whose expense? These are serious questions, which the general public has been plainly ignoring until recently- and that's being entirely overly-favorable.

I think the real harm here is that someone like Taylor Swift can have her searches scrubbed. What about Jane next door? She has no such recourse.

It’s actually the opposite. X is making these changes on their own. An individual out of the spotlight could probably sue.

In the law public figures are not protected the way individuals are. When sell out and willingly make millions of dollars exploiting your likeness, you don’t get to choose if people make a parody of that likeness.

Yes, X is doing it because Taylor Swift is an important figure for their platform. Jane could sue, but X would claim no responsibility and there would be no loss to X if she left the platform.

Well I am going to bluntly disagree, sorry. I am dismissive. But in the end, I will be right and you will be wrong. That's the pattern in technology.

1. Some technology is invented 2. It changes the status quo, directly or indirectly, on society. 3. There is some degree of backlash, regulations, and outrage. 4. Everyone accepts the new reality and forgets what it was even like before. Step 3 is often forgotten and considered quaint.

The reality is this is an unsolveable situation. Porn deep-fakes are here. There is nothing that will change that. 20 years from now, celebrities will just accept that they are going to exist.

I'm not even arguing that the technology is good (im not sure it is) or that I support it.

I am arguing that it's inevitable.

You think I am going to be irrelevant, but I think it's the opposite.

In the last century we have invented number technologies responsible for megadeaths (cars, air pollution, carpet bombing). If those couldn't be stopped after killing thousands every year, then neither will this.

Your step four is incomplete. In addition to accepting a new normal, we put limits on technology and address the harms. We didn't roll over and accept, say, the destruction of the ozone layer. We put restrictions on the use of CFCs, and we invented new refrigerants that were much less harmful. Though there has been some recent backsliding due to rogue emitters, these measures were effective at stopping and eventually reversing the degradation of the ozone layer.

Some more examples include unleaded gasoline, breaks without asbestos, paint and cosmetics that don't include heavy metals, light bulbs that are much more efficient, etc. We've also figured out how to make fuels that don't contain all much sulfur, thus avoiding issues with acid rain.

This is an attitude I see on HN a lot. Technological innovation is seen as unstoppable and all powerful - until it's time to innovate new ways to address the problems we've created, and then there's apparently no possibility of coming up with innovative solutions. We're to accept ourselves as passive actors, caught up in the currents of history, with no agency to alter course.

An example that I find haunting is OceanGate and Stockton Rush. Rush was all for being innovative when it came to novel submersible construction. But when his engineers wanted to inspect the hull for degradation - can't be done, the hull is too thick to scan. You might expect someone who views themselves as a fearless innovator to see that as an opportunity to develop a new methodology or technology. But Rush was only a fearless innovator when it pleased him, and so he took himself and his customers to their deaths.

Frankly I think this attitude is a bad excuse, and not one that society will tolerate forever. This is the path to crippling regulations and trust busting. We can't just foist our externalities on the rest of society forever and not expect them to eventually decide we're incapable of self regulating.

I can't help but to see a parallel to the 90's crypto wars and end to end encryption debate. The technology and models to make these fakes is already out there and are freely downloadable by anybody, and unlike asbestos brakes or incandescent light bulbs it isn't going to naturally expire at any point.

Even if the most crippling legislation was passed to ban such technology from law abiding citizens, criminals would still have their E2E messaging and the deepfakers would have AI at their disposal. Hate to be a doomer, but the smoke is already out of the box, what can you do?

I didn't write this because it was too long already, but it depends on the technology.

I agree that CFC bans are a good example of when something was done.

But I can also name many situations in which nothing effective was done. And even in the face of overwhelming effort to halt it, the resistance amounted to little difference.

The difference all comes down to the technology.

So, my argument is that there are some technologies which can be addressed, and others which cannot.

Solveable problems tend towards those with few decision makers, low incentive to defect, adherence can be monitored, costs are internal, and where the problem exists at top-level of large organizations.

Unsolveable problems tend towards those with lots of decision makers, high incentives to defect, difficult to detect violators, costs are external, and the problem is personal.

CFCs fall more in the the former--there just aren't that many CFC manufacturers (apparently 18 companies accounted for > 99% of production). Consider for a second, an alternate world where everyone builds their own refrigerants in their garage, and each individual decided whether to use CFCs or an alternative. Do you think we would have solved CFCs by now?

This particular issue is very much the latter. The tech is already out in the open. The decision to use it responsibility or not is left to individuals.

You mentioned foisting externalities on the rest of society. I agree that this is happening. And I agree that society will not stand for it. But my point is not that society is going to accept it, it's that society cannot do anything about it. Oh, people will shout and stomp and raise hell. Hearing will be had. People will be lambasted in front of congress. Voters will decry politicians to do something. And efforts will be attempted. But in the end, it will all be for naught.

I realize this is a doomer outlook. Please don't mistake what I am saying as an endorsement.

Here's a scenario:

It's 2030 and deepfakes run rampant. This is a problem creating political deepfakes, celebrity fake porn, CSAM, fake revenge porn, etc.

Apple builds on their Spatial Photo feature, and their newest smartphones allows POR, or Proof of Reality.

Proof of Reality: Data from multiple cameras and a LIDAR sensor are stitched together to generate a 3d depth-map + color map that can validate that a photo or video was shot without post-production manipulation. This processing is done on-chip, on a Secure Enclave-like chip, and cryptographically embedded in the following photo. Each raw capture starts as the first block of a blockchain; further images or videos that are created from this raw data are understood to be downstream of this first block.

A piece of Proof-of-Reality media, edited together with multiple clips or images, can be cryptographically verified that it is composed out of individual Proof-of-Reality media. Like a Merkle tree.

Apple pioneers the first fully Deepfake-proof media workflow. Consumers can watch news media or social media while being cryptographically assured that it wasn't AI-generated.

2031: Proof-Of-Reality (POR) starts to catch on in public. Samsung gets on the bandwagon, and develops their own version (or joins a POR consortium). Soon, 40% of media is POR-validated, following the usual smartphone & OS update statistics.

2032: A particular egregious deepfake scandal from a non-POR source drives the rush towards POR standardization. Apple and/or the POR consortium partners begin to produce more professional-level POR camera equipment. Content blockers that block non-POR media become developed.

2033: Certain social media websites begin to place labels notices on non-POR media. POR-media consists of 70% of all news & social media.

2034: News media companies fully switch over to a POR-workflow. Browsers start adopting non-POR labels for content, like Twitter's 'Community Notes'.

2035: Deepfakes as we know it are mostly hidden from the public eye, but continues to evolve and change in unexpected ways..

I love it! Though I am not an expert, I can at least see this.

However, I hate to be a nitpicker, but I think this is solving a separate issue. I don't think this issue is about authenticating the legitimacy of deep fake porn. Rather, the mere existence of it is the issue.

That is, people don't care that it's fake. People don't want to buy the Apple proof of reality because they don't want reality.

Good point! Then:

2036: Due to increasing amounts of deepfake CSAM, the US's Congress passes a law against unconsensual deepfake porn, requiring "websites of sexual nature" to be POR-compliant or be shut down. Porn web companies, ISPs/hosting providers, and credit card processors alike are legally liable.

Pornhub welcomes this change with cheeky 'PORnhub' branding, but the reality is that change is necessary or they will be sued out of oblivion.

Prosumer platforms like OnlyFans welcome POR-validation with wide arms, because it bolsters their image of authenticity. Exploiting the ban on deepfake porn, "softfake porn", where celebrity look-alikes create porn, becomes mildly popular.

2037: Eventually, ISP / hosting providers / credit card processors that instigate the change. Much like SESTA/FOSTA's impact on sex workers in the early 2020s, payment processors and ISPs refuse to work with POR-unvalidated porn sites. Eventually, porn sites shift towards POR-compliance, and create new niches.

Of course, underground deepfake porn still exists, if you know where to look. But by now, its associated reputation with CSAM makes it very inaccessible and disdained.

Gone are the days of rampant deepfakes in the late 2020s and early 2030s. Mainstream media and politics call this a success, but a minority are angry, saying that deepfakes are a creative act, and the effective ban on POR-noncompliant material is a further restriction on creative liberties. ..

2030: Deepfakes are rampant, causing significant issues in politics, entertainment, and personal privacy. However, instead of technological solutions, there is a growing trend of regulatory capture. Large corporations and governments begin to argue that deepfakes are an inevitable part of the digital landscape. The cost-effectiveness of creating deepfake content compared to traditional media production becomes a significant talking point.

2031: As deepfake technology becomes more sophisticated and cheaper, it starts to replace traditional media production methods. Major studios and media companies lobby for and receive regulatory approval to use deepfakes as a legitimate form of content creation. This shift is justified by the reduced cost and logistical ease of using AI-generated characters instead of real actors.

2032: A scandal arises involving a particularly damaging deepfake, but instead of driving a push towards authenticity verification technologies, it leads to further normalization of deepfakes. The argument is made that since distinguishing between real and fake content is increasingly difficult, society should adapt to accepting deepfake content as a new norm.

2033: Social media platforms and news outlets begin to openly embrace deepfake technology, citing cost reduction and the ability to generate more engaging content. Traditional media actors and creators are increasingly marginalized, with deepfake creators dominating the market.

2034: Regulatory bodies, heavily influenced by big tech and media conglomerates, begin to actively promote deepfake content. New regulations make it easier for deepfake content to be produced and disseminated, while traditional media production is bogged down by increased costs and regulatory hurdles.

2035: The public gradually accepts deepfakes as the primary form of digital content. Traditional media, with real actors and genuine locations, becomes a niche market due to its higher production costs and complexity. Deepfakes evolve in unexpected ways, permeating every aspect of digital media and blurring the line between reality and AI-generated content.

I understand you are not offering an endorsement, and I appreciate that we can disagree without being opposed.

I think our disagreement is quite narrow. I subscribe to the view that everything in history is contingent and nothing is inevitable. I gather you believe some things are inevitable, and that the particulars of a technology might make certain things inevitable. Thank you for elaborating.

Sometimes we're not able to mitigate the harms of technology; fossil fuels come to mind. But I would argue this has next to nothing to do with fuel technology. It's because there are power people in our society who have successfully slow walked a transition to alternatives. But they haven't managed to stop it altogether - solar, wind, and EVs are seeing widespread deployment.

The tech industry has been able to do this to some extent, but I don't think they have nearly the political capital of the fossil fuel industry. We're seeing increasing calls in the US to regulate the tech industry and to break up the larger companies. While the two parties aren't able to cooperate on nearly any legislation at this time, this attitude is bipartisan. The EU has already begun to regulate the tech industry. (I can't comment usefully on the rest of the world.)

I do agree that the properties you enumerated make a problem easier or more difficult to solve, and that there's no way to stop people from making deepfake porn altogether. But that doesn't mean there isn't anything to be done about it, even if you and I might not be smart enough or positioned correctly to understand what that might be.

My goal is to convince people not to give up on the idea that something could be done, and not to become entrenched in a certain position. Eg, if someone doesn't like that X responded by disallowing searches, I don't want to have yet another argument about whether or not this is a free speech issue. I want to hear that person offering a critique I haven't heard before, asking interesting questions, suggesting bad or partly formed ideas about what to do differently. And I don't want to see the rest of us start a flamewar beneath their comment, I really to see the rest of us pointing out weaknesses in a proposed solution and how we might address them. I want to see this community being creative and constructive about this, and I believe that really could make a difference.

If we set aside doomerism, the worst thing that can happen is that we're wrong. That's not so bad; let's chance it.

Ohhh nooo...A celebrity got mocked online...how horrible!!1I! /s

I assume the deep fakes were mild compared to what goes on in some of her fans' minds.

I wouldn't say she was mocked, I would call this harassment. You don't stop having rights when you become a celebrity. I also don't believe it's safe to assume this sort of thing will only happen to celebrities going forward.

I think you know the difference between imagining something and publishing something, and that you're being facetious.

Send me a face portrait and the address of your employer and some of your friend group and I wlii mock you as well. Just so you personally experience it and see what a load of shit your comment is.