[1]: https://www.theverge.com/2019/10/22/20927521/snap-growth-ear...

Also, I'm fascinated by the idea of using AI to detect the use of AI. How good will your deep fake have to be in order to fool a deep fake detection AI?

But recently I read "Tlön, Uqbar, Orbis Tertius", and I thought that the point Borges was making could be easily described as critique of "fake-news post-realism of today", except it was 80 years ago. And while the notion of this problem not being new at first seemed weird to me, eventually I figured this is pretty natural: while 80 (or 800 for that matter) years ago there wasn't any such tools to manipulate information on that scale, nobody had the access to information on that scale in the first place. So anyone could be fooled by more powerful actor anyway. Offensive tools of any time are roughly the same as the defensive tools.

It's just that these 20 years at the turn of the century were somewhat the sweet spot, when we had this "Internet for people like us", so it was us who was fooling the less knowledgeable people, while keeping the belief that we are relatively safe from being maleficently disinformed by somebody else. And now we can clearly see it's not necessarily the case anymore, ao it feels like something new.

But it really isn't. As always.

The best generator and the best detector are actually part of the same model! If you create a better detector, you are making the generator better at the same time so you have not accomplished anything.

"GANs are a clever way of training a generative model by framing the problem as a supervised learning problem with two sub-models: the generator model that we train to generate new examples, and the discriminator model that tries to classify examples as either real (from the domain) or fake (generated). The two models are trained together in a zero-sum game, adversarial, until the discriminator model is fooled about half the time, meaning the generator model is generating plausible examples."

https://machinelearningmastery.com/what-are-generative-adver...

And it does seem that the human-visible weaknesses of many currently published deepfake examples are those that would be hard for the current discriminator architectures to detect, not those which would be harder to generate than any other part of the fake.

1) The researched human behavior where "1000 lies must be equal to 1 truth".

https://www.researchgate.net/publication/256486791_Credibili...

2) Monopolizing the information stream: the attention grabbing effectiveness of deepfake videos to convey fake messages and its novelty incentive to share them on social media, added to the fear-of-missing-out they generate on viewers and the fact that "the effort required to refute b* is 10 times greater than the effort required to produce it":

https://www.telegraphindia.com/india/social-media-fatigue-an...

3) voter apathy caused by news fatigue due to the sheer volume of fake news, to which deepfake adds a new vector:

https://www.nytimes.com/2019/11/18/us/polls-media-fake-news....,

https://www.datadriveninvestor.com/2019/07/04/chronicle-fati...

4) A majority of youngsters can't really spot (or care) about fake anymore:

https://qz.com/1750839/most-young-2020-us-election-voters-ca...

The film "Mr. Smith Goes to Washington" came out in 1939. It should be mandatory viewing.

It's won't upend the world, but we are losing a hugely helpful tool that has been relied upon for 60 years.

The technology to fake paper ballots is trivial, but they still (mostly) work fine for elections, because we've developed strict rules for how they need to be handled in order to retain credibility.

Sure, and that's part of the danger. Convincing deep fakes will be used to delegitimize mainstream news organizations, as with what happened with Dan Rather - a trusted source presented faked documents.

The people who care about the 'legitimacy' of a news organization are moderates who are passively engaged and are being rapidly driven to the margins. The 'objective truth' of the matter is less relevant than how it is received. Activist politics is en vogue thanks to the feelings of helplessness brought on by globalization, and if you're not an activist your views are not really an important part of the process anymore. You'll have to pick one of the two options that the activists come up with because they won't show up to vote for a centrist candidate. In either case, a moderate platform cannot win.

For the record, I feel the above situation is incredibly dystopian; but we're hardly the first country where pathos supersedes logos and ethos.

If 2004 news could be fooled by a fake that took a couple of minutes to make, I’m not that worried that today’s news is any better.

And we've been hit by tabloids, spam and so on before. None of that is new. Overwhelmed by content is/was/will be a problem that is solved very simply: people limit the distribution lanes they consume, and naturally establish "trust networks" and hierarchies.

Nyt didn't become "trusted" by accident. People aren't as dumb as you seem to think.

Anyways the problem is of distribution, not that a lie can be told. If we can't trust our distribution lanes to actually reflect the institutions we're trusting, then we can't establish our trust networks.

And this is a real problem: our distribution lanes are divorced now from our actual information distrubuters; medium, Facebook, Twitter, etc fuck around with our trust networks, randomly injecting their own bullshit into our feeds and messing around with feed-order based on non-trust metrics (eg money paid) such that they've become fairly unreliable.

And our classical trustable institutions have become less trustworthy, as they flounder about trying to make sense of the "digital age", and have so far done so in a pathetic fashion

When that network breaks down, you only need a single good lie to do the damage.

This is, itself, "fake news". I can't find any indication that other mainstream media outlets defend the documents "to this day". The reporting largely seems to be "certainly smells like bullshit, and we can point to specific issues with them that seem to support that".

https://www.nytimes.com/2004/09/13/opinion/those-discredited... cites a wide variety of news organizations looking into them and finding problems, stating "When the mainstream press checked the sources mentioned or ignored by '60 Minutes II,' the story came apart." and noting Newsweek specifically calling them "discredited".

The only difference is today, they don't retire out of shame, they just keep on doing it, keeping viewership via emotional draw rather than rational news. Real news has been dead since that incident in my view. It's mostly state propaganda, regardless of party.

"deepfake news"

I can just hear people saying it now. Joking aside, and not taking in ANY ethical/unethical misuse considerations this software is capable of, it is really cool tech.

"With software there are only two outcomes: either the users control the program or the program controls the users. If the program is not under user control, and the developer controls everything, then the program is an instrument of unjust power, applied by the powers that be."

Deepfakes are just a continuation of "fake news" (formerly known as propaganda), and the end outcome is we either succumb to those who use it, or adapt - filtering out the bs isn't easy, but it's not an insurmountable task, either.

And what about when this happens at a state-sponsored level (This is absolutely already in development, if it hasn’t happened already...). “Did you see that Fox News exclusive video leaked from the Bernie Sanders campaign? Yeah he was holding a private dinner for a bunch of Billionaires. They say they got it from some Russian oligarch he invited. So much for grass roots change, I think I’ll just stay home this Election Day.” Or the corollary: “That smoking-gun tape the Do-Nothing Dems have of me at Putin’s personal sex trafficking center, where the two of us are laughing about having Epstein take the hit? Fake news! They’re called deepfakes, and they make these just absolutely incredible fake videos. Not real at all though. Only trust what the Government-sanctioned media says!”

As per your other point, given the GANs that are used to make these are precisely a deep fake AI fooling a deep fake detection AI, aren’t we already there? Further, when the government is making the generator, a privately trained discriminator has no chance. How would you even train that? Get the government to give you a bunch of generated fake media so that you can develop a really spot-on model? I don’t think so.

Two predictions:

Deepfakes will play a roll in the 2020 election.

Deepfakes will play a roll in every election thereafter.

If this doesn’t concern you, I don’t know what would. The thing with the “post-truth world” is that once people think we’re in it, we are.

Long have documents and news been the subject of fraud. And for a few decades photo alterations have been common. Now we are adding video to the list.

But there is nothing new about the faking of libelous material (eg Killian documents 2004)

How many people do you think actually read those “Killian documents”? How many people do you think a state-sponsored social media campaign built on the most accessible media would reach?

Are we though? Or is it just becoming much more obvious the limitations of relying on the media in general. This very well might lead to a better understanding of truth, and our personal responsibility in understanding it.

At the very least, it's not a foregone conclusion that this will lead to worse outcomes than what we have today. It might be what tips us in a new and better direction.

What we'll continue to see are rising sects of flat-earthers, anti-vaxxers, cultists, polarization and deniying of anything not-in-my-bubble, a complete lack of trust in journalism and media, and futile attempts of fact-checking while we're all drowning in fake news shared endlessy on unsuspecting users. It takes a lot of effort to show something was manipulated digitally, and you'll have to fact-check by yourself, that doesn't scale at all.

The fact that we're more aware of flat-earthers today, is probably a good thing too. I'm not 100% sure that there are more of them today than before, or if they're just more noticeable. But in either case, we should be very careful to draw too many conclusions when we look around today because we're living in a time of great transition. We're basically in our teenage years of this technology.

Just as you shouldn't judge an adult by who he was at 14, I don't think we should assume we know what the adult internet will look like because we see some acne on it today.

Anywhere you have trust, there's a potential for abuse. A seemingly natural response to abuse of trust is to find a lapse in the victim's vigilance and start strapping on your own armor. I listened to a podcast this fall (sorry, I don't recall for certain--my best guess is that it was peter pomerantsev on Ezra Klein's podcast) where the guest suggests that some institutions/relations (like markets) depend on trust to produce value and may be undermined or degraded if enough participants take an individualist caveat emptor approach.

It is hard to know, in a complex adaptive system, what new equilibria will emerge downstream from a big disruption. We certainly could emerge through the other side with better epistemologies. But I don't see it as a given. Thinking is hard. Proving anything is harder. Skepticism can be good (and I suspect we need more--though I don't think it's terribly well distributed or applied at the moment...), but it's also easy; it requires no trust, knowledge, thinking, rigor, logic, or proof.

So, I'm troubled by the possibility we'll dig ourselves into a deep world-shrinking skepticism (a callous) of anything we didn't/can't personally experience. It can be well-founded on real abuses of trust. It may even be the sum of individual rational choices.

Neither would necessarily keep it from unmooring things we mistook for solid ground. And, when someone finally notices one drifting, there may not be enough trust left to agree on fundamentals like: is it adrift? is that a problem? was it ever tied down? is there anything we can do? should we wait for a bit to see if it drifts closer first? should we act?

But i'm terrified of the current loud support toward censorship and authoritarian suppression of incorrect ideas. We have a lot of historical evidence of just how bloody that always turns out. So I choose to believe in the power of openness and competition of ideas.

As others have posted above, we will never have a time where it's possible or desirable for each of us to validate the truth of every claim, so that means we will have to find trustworthy arbiters of truth. I think that this should happen by people showing they are trustworthy and earning respect, rather than using authoritarian control to shut down those with differing ideas.

Much of the media today has lost such respect, and deservedly so. You can not fix this by suppression of free-speech, but only by creating honest organizations that aren't afraid to respectfully show the ideas of those who see things differently -- even if it's just refute them. Organizations that _trust_ the viewer to decide between fairly articulated positions.

The contempt for our fellow man when we say that bad ideas must never see the light of day or enjoy the same platform as government-approved ideas is a deeply paternalistic and ultimately anti democratic idea. It's elitism at its worst.

(For example, we need to be able to trust that they have a sincere opinion and aren't paid shills for a corporate lobby on a PR mission to waste our time, emotional energy, column-inches, and attention.)

If you feel a growing sense of dread as it retreats, it's a good time to consider whether you know others who have expressed a similar sense of dread that other things in our world have come unmoored, and whether they could use some help keeping them tied down.

Even society is shown to be untrustworthy, and the social processes of science.

Thus some reactionaries want to replace trust with authority. Or markets, which is another name for social proof, but with much less transparency.

Ultimately the other parts of trust are verification and reputation, which are both not unassailable. (Sybil style attacks, bribery, institutional and social biases.)

But, our saving grace might be simpler: cooperation without trust is possible and might be as efficient if things get bad enough.

Even this is too simple. Many people have lived under propaganda regimes, yes, many have bought into them, and many people have seen through them.

Each new wave of information technology provides new ways to “pull the wool” over peoples’ eyes.

And though a propaganda machine has power... like all things it creates and equal an opposite power that eventually destroys it. The question for individuals is only: where am I in the cycle, how long will this cycle last, and what is my role to play?

The thing is, people do not check information that matches their viewpoint for validity. Because being wrong feels bad.

None of that actually happened because people learned to be skeptical, and to care about who took the photo, the chain of custody, and the controls to prevent digital alteration.

Maybe we need something similar but more rigorous for video. The camera itself could encode a PKI signature in every frame of the original. If subsequent edits preserve frames as is, then the viewer could determine how many frames were cut or duplicated.

If subsequent edits crop, recolor or otherwise alter the frames, that editor could encode its own digital signature so the viewer could judge how much to trust their edits.

Is there evidence that deepfake videos will be worse than photos? My (naive) sense is that even for an AI, editing a photo will always be easier than editing a video with sound.

If Photoshop and deep-fake images aren't a problem, shouldn't we be less concerned about video?

This may be true in courts of law, but it absolutely is not when it comes to general public consumption of pictures.

The 9/11 Tourist Guy picture that duped a lot of people 18 years ago looks like a ridiculous MS paint edit today. https://en.wikipedia.org/wiki/Tourist_guy

Photo editing has been used pretty much since the beginning of mainstream photography. You should never "trust" pictures for anything serious if you don't have absolute trust in the source.

Besides it's one thing to edit a still image and a whole other to forge a long video, complete with audio. Look at those movies these past few years using 3D models to insert younger versions of actors (Terminator and Star Wars come to mind). They must have invested millions into it, they have very talented artists and it's pretty good but it's still quite easy to spot the fakeness of it.

The Daily Mail was at it from around WW1, perhaps earlier. Though their main WW1 fake was, of course, the cadaver factories. Surprisingly the Times bought that tale too.

Likewise, a deepfake is worth a thousand lies.

And why doesn't the government keep track of the cameras and security keys so they can trace back any images they do not approve of?

Maybe I’m being paranoid but it seems that the public is being conditioned to believe deepfakes are so good prior to a certain cache of video recordings being made public