> merely the next iteration of Photoshop

I think you're significantly understating the impact here.

You're talking mostly about content-creation, and in this context there are a lot of positives; I certainly agree with your statement:

> They're going to breathe new life into media and make it significantly easier to make films, music, and all kinds of art.

BUT. I don't think you can, with a straight face, say that there are going to be no negative impacts of this technology. You make a claim that this is just like Photoshopping in the 90s, but I think the key difference here is that in the '90s, there was a fallback medium; when photos became unreliable, you still had video, which couldn't easily be faked.

Looking forward to the next 5-10 years, how does anyone figure out what's true when photos, video, and voice can all be trivially faked by anyone? What reliably medium can we fall back to? This could lead to a real epistemological crisis!

> The public will come to understand AI-generated media as a part of daily life. People are not that stupid.

My problem with this position is that it hand-waves away some pretty fundamental changes to the fabric of our reality. How exactly do you propose that people are going to understand the world, post-'fakes? If any video you see in your twitter feed could be fake, how can anyone make any inferences about the world outside of their own immediate direct experience?

There was a time before video, before photographs, before writing even.

So, not new at all. We've got thousands of years experience in dealing with truth under limited circumstances with false claims. Consensus reality was a passing fad.

Video of the same event from multiple angles will probably be safe for quite some time (timing and lighting mismatches

Video from a single source already has the issue that the person showing you the video can start and end the video at whatever time they want to.

You say that "People are not that stupid" but have you looked at all the obvious fake/false information being passed around on facebook. I think that shows that people really are that stupid.

I think it shows that nothing about Photoshop or deepfakes matters in the slighest since the kinds of content that people are falling for now is a stock photo with some text under it.

Seems to me that that the end of your comment supports the parent commenter's point, rather than showing the irrelevance of deepfakes for the soundness of public opinion — that people are so easily and constantly deluded now gives little reason to hope for a discerning public in the face of stronger fakery

Much of that misinformation is also made with far cruder tools than a well-trained GAN. People will be fooled by what they want to be fooled by. Remember how Republicans thought Nancy Pelosi showed up drunk to some political event because someone just slowed down the video? Or the whole wave of cleverly-edited "expose" videos on ACORN and Planned Parenthood done by the Project Veritas guys? The reality is, anyone who believed those videos already had a negative opinion of those organizations and people that jived with the lie they believed.

It doesn't matter if Nancy Pelosi wasn't actually drunk, she's a lunatic anyway!

It doesn't matter if Planned Parenthood wasn't actually selling body parts for profit, they're baby killers anyway!

It doesn't matter if ACORN wasn't actually telling pimps how to file taxes, they're Obama lovers anyway!

The problem is not that it's possible to convincingly falsify video evidence. That's the kind of problem you need to deal with as the judge of a courtroom, and courts already know how to deal with manipulated and falsified evidence in other contexts: strong monetary and criminal penalties. However, we're talking about the people at large, who are not unimpassioned triers of fact but people who have strong opinions that they are angry about. The court of public opinion is not an impartial or fair court, it is a cacophony of angry people shouting into the void until society gives them a perceived win.

Look at QAnon nonsense: 4chan convinced a good chunk of conservative America that Trump is fighting a global child cannibalism ring by just writing a bunch of incomprehensibly long-winded stories. People aren't stupid, they're angry.

It's funny you mention the court system as knowing how to deal with manipulated and falsified evidence, but you imply the videos from Project Veritas, &c., were "cleverly-edited exposes," when it is the courts - and even Planned Parenthood, while in a legal case - that have found those videos to be factual, and to not have any particular omissions, and to be valid evidence:

https://www.texasattorneygeneral.gov/sites/default/files/ima...

page 10:

> The district court stated, inaccurately, that the CMP video had not been authenticated and suggested that it may have been edited.

> In fact, the record reflects that OIG had submitted a report from a forensic firm concluding that the video was authentic and not deceptively edited. And the plaintiffs did not identify any particular omission or addition in the video footage.

[Planned Parenthood are the plaintiffs in this specific case.]

A bit more information on this subject: https://www.youtube.com/watch?v=POdbu4bp5rQ

Including some of the videos referenced themselves: https://www.youtube.com/c/TheCenterforMedicalProgress/videos

Re your QAnon comment at the end -- I'm not sure if you saw, but it seems that (as of today) people are defending Q on HN as well :( Is disinformation against HN policies @dang, and if not, shouldn't it be?

I think this analogy is too simplistic. Video fundamentally has a level of engagement and connection to human senses and perception of reality that goes far beyond photography. By the time that fast/convenient photo manipulation tools became widely available (i.e. not Photoshop 1990-2010, but well into the 2010s), video had already long replaced photo as the average human's gold standard for evidence.

Even just thinking about the past 5 years (or even just the past 5 months) think of how much impact citizen phone video has had just in the U.S. – outside of a courtroom. Even well before George Floyd, the power and utility of bystander footage has arguably been the strongest factor in public advocacy for state and local police to mandate officer-worn body cams.

While what you say is reasonable given a sufficiently long span of time, I think it's also reasonable to expect that there will be very significant growing pains between now and when the public ultimately comes to understand AI-generated media as a part of daily life.

I think we're maybe a year away from that.

Most of these results are being produced from open source models that are difficult to set up and run.

I joined a few Discord servers that were started this year for deep fake video and voice, and they're now huge. We're also just now seeing subreddits explode with content [1]. My website, vo.codes, is starting to take off on Twitter. My phone is blowing up with retweets, mentions, and followers. Youtube is overflowing with the stuff [2]

Give it a year and these results will be everywhere. The kids love it. Memes are going to normalize it.

[1] https://www.reddit.com/r/mediasynthesis

[2] https://www.youtube.com/watch?v=hRlo8_13Zyw

I think you and I are there now. But there's a looooong gap of knowledge between you and me about this stuff versus, say, grandma and grandpa who live out on the farm.

To be clear, I'm not a luddite, I think this tech is/can be extremely fruitful for creatives -- a new paradigm of creativity. But the fruit can also be plucked by many propagandists, and IMO there can be serious consequences right now unless we educate people about this stuff right now. I'd argue that the approach can't be fear alone, but it also can't be all sunshine and rainbows.

Edit: I also just want to say that the work you're doing/referencing is very helpful toward this education

I'm not trying to be adversarial, but you must truly live in a bubble if you think that the average person will be able to catch on to all the ways this can and will be abused. There are already huge problems with people being influenced by very obvious botnet trolling. It has serious political implications. You're probably just not around or in the social media sphere of the kind of people who are into e.g. flat earth conspiracies.

I am curious about how this will affect advertising and e-commerce. There was a post about bots generating cosplays and clothing in real time for purchase. That seems more interesting usage of this tech.

Motivate people to buy something by showing them personalized video ads of someone they idolize calling for purchase?

Facebook has all the data and network it needs to pull deepfake advertisement and shopping business.

If you are grounding your thoughts in the expectation that "people are not that stupid" you are setting yourself up for failure.

> Gilbert Gottfried

How much would you charge to not make it easier to release more of that nails-on-chalkboard voice into the world?

I know HN isn't the place for flippant jokes, but I would literally pay for Gilbert Gottfried narrating serious things like Planet Earth. I think it would be amazing.

https://vo.codes has you covered. You can also use David Attenborough's voice to narrate Gilbert Gottfried standup.

> People are not that stupid.

We act as if there has to be some majority of opinion for anything to be impacted.

> Bruening miscalculated the mood of the nation after six months of economic depression. The Nazis won 18.3 percent of the vote and became the second largest political party in the country.[1]

> Roughly one-third of Americans who have heard about it see truth in the conspiracy theory that the COVID-19 outbreak was intentionally planned by people in power [2]

We're still fighting a conspiracy from the Middle Ages.[3] And it has overt political support again.[4]

[1] https://encyclopedia.ushmm.org/content/en/article/the-nazi-r...

[2] https://www.journalism.org/2020/06/29/three-months-in-many-a...

[3] https://www.wired.com/story/opinion-the-dark-virality-of-a-h...

[4] https://www.mediamatters.org/qanon-conspiracy-theory/qanon-m...