We will eventually find black-market "off-the-grid" data centers where you can run AI algorithms away from government-mandated watermarks, and generate untraceable content to topple politicians and such.

You're spot on, I think. This is already happening, and disintegrates society. There are countries where an analogue version of this is already implemented: fake parliament with fake opposition, fake government, fake media, fake prosecutions, fake health care system, fake education system, fake state functions. By fake I mean: it looks like a real one, but it's real goals and purposes are very different from what is officially advertised.

It's scary to think about how AI will boost the already way too effective politics of "fake".

Which countries are you referring to, with fake versions of all institutions?

I live in Hungary, where I have this feeling of "fake state" getting stronger and stronger every year. I'm sure there are other similar countries.

One recent example: our education system has been neglected for long. Now, that we have an inflation of ~25% (inflation of food is around 50%), teachers literally can't make ends meet. They started to fight for themselves, and instead of taking the problem seriously, the government fights back with its power. (E.g. by firing or silencing teachers who demonstrate.) Teachers are leaving for other jobs in huge numbers. The buildings of even some of the best schools in the country are in catastrophic shape, on the brink of causing major damage to those inside. All this, because it is not a real priority to have good schools. This is only advertised, but it is a lie. The whole education system is gradually shifting into a mode of "baby sitting" kids while the parents work.

Another example is the prosecution system. Interestingly, they are very quick and effective in investigating the smallest wrongdoing if it helps those in power. If the investigation would hurt those in power, they very quickly abandon the investigation with funny and obiously fake reasons. Again: the prosecution system looks like a real one, but it's not. It has purposes different from what is officially advertised.

The closest advisor of the prime minister openly said this week, that "if you control the media, you control the thoughts of people". This, sadly, seems to be true. It really seems that the point of the government is not to run the country decently, but only to fake it. And it works.

Or maybe people will just start requiring more evidence that something actually happened instead of believing everything they find on Tiktok (or the wider internet), which would actually be a good thing.

Video evidence may not be trusted as evidence in court in the future. Will we require something more perhaps? Or will the word of AI keen on spotting something's off be what we trust?

Has forensic analysis been defeated by a deep fake? At least in the courts cases won't depend on the jury being able to 'tell by the pixels' and instead experts will get called in. Whoever that ends up being, if they can do their job and authenticate a video at least as well as they already can with photoshops we should be fine.

Photoshop didn't cause society to collapse and photoshop-for-video won't either.

>and instead experts will get called in.

the defense often cannot afford experts.

Also there is a long history of experts lying, in the U.S at least but I suppose elsewhere as well.

That's kind of encouraging then. The lying experts and underfunded defense attorneys out there didn't bring about the end of days with the appearance of photoshop, and it's unlikely that they will now either. Which isn't to say that we shouldn't try to fix those problems...

I'm already somewhat hopeful that if deepfakes can be reliably detected juries will be automatically skeptical of unauthenticated videos and prosecutors will view getting their videos authenticated as genuine as an easy way to strengthen their case. I'm not even on a jury and I suspect close to every picture I see has been edited and altered somehow.

Maybe we will only trust video from cameras with embedded cryptographic functionality when the cryptographic checks verify it hasn't been tampered with?

(And even then sometimes wonder if a hardware hack was involved)

What evidence can the average person verify?

Historically, the average person could verify very little. To this day I've never once seen China with my own eyes. Does it even exist? Probably... how do I know? We all decide on others we consider trustworthy enough for the context.

If some major news org said they contacted Tom Cruise and asked him if the video was real and he said it wasn't him, then as an average person I'd probably believe them because I have at least some degree of trust that they'll either tell the truth or get called out on the lie and ultimately what do I care either way.

It's extraordinary claims that require extraordinary evidence and "Paris Hilton and Tom Cruise do rich people things on camera" doesn't meet that threshold. If someone posted a video of the president eating a live baby I'd probably be more skeptical of my sources.

> or get called out on the lie

Get "called out" how, exactly?

With the distribution channels centralized, how do you call out a lie by a "major news org", if the gvt (Google, Youtube, FB, Twitter…) decide to play along and suppress your message?

The EU Commissar of Truth, Vera Jourova, announced that "the era of the Wild West for free speech is over". She also cited "gentlemen's agreement" with the "big boy" platforms where legislation is lacking for now. Chilling.

To the degree that the censorship machine fails, it fails for technical reasons, not for a lack of appetite.

A resigned shrug and "ultimately what do I care" is increasingly common. It might be a self-correcting problem, on the evolutionary scale.

> If someone posted a video of the president eating a live baby I'd probably be more skeptical of my sources.

Isn't that because you've lived a life grounded (mostly) in physical reality? With free access to (mostly) uncensored information?

We might be surprised what's considered "extraordinary" with respect to a "sceptical threshold" in the future, especially once the pre-internet generation dies out. And digital information with digital influencers (AI or not!) beholden to a few centralized platforms shaping the consensus reality.

Whats in it for the proverbial me to do that? I can see something and accept it if it aligns with my world view, or spend time and effort trying to disprove something that might actually be true - and how can I know the proof isn't the fake?

9/10 people will just accept it, probably 7/9 wont have much other option without any practical ability to authenticate the hundreds of things they see a day.

Another head-in-the-sand answer with some vaguely related talking points that are utterly unbelievable. Maybe pigs will fly tomorrow. Who can tell?

I'll give you one decent example from recent popular media: Sandy Hooks. Alex Jones didn't even need a deep fake to convince people.

Before Photoshop, people believed in manipulated static images (as seen by all the fake UFO "photographs"). After Photoshop, it required "video evidence" to make people believe in UFOs again. Once it's possible to easily fake video material, the same will happen as with doctored static images. People will stop believing in video without additional evidence that the video is actually real.

PS: People being manipulated by propaganda is really old news. The only thing that the internet has changed is that all the village idiots who easily fall for propaganda discovered that each village has its idiot and they started to communicate and coordinate. But that's unrelated to deep fake and was already a problem before.

ok well once you don't believe in video evidence or photographic evidence then there is no evidence that could convince you of something other than actually experiencing it yourself. In which case people will just believe what they want to believe.

Usually there are other witnesses, other video sources. The more material exists, the harder it is to make the fake believable. You have to consider the sources. Who published the video, what are their motivations, who pays them.

Really, that's "internet user 101", I can't believe we're having such a discussion on HN :)

Or more likely, people will require essentially no evidence for something that confirms their bias, and an unsurmountable unreasonable body of evidence for something that contradicts it.

Given what happened to humanity in the past decades, I think this is completely shatter trust in everybody and everything. Once the first large news outlet mistakes a deepfake video for a real one and drops a big headline on it, it's only going to get worse.

Faking identity is just one side of it, I actually think we are still relatively safe, when we get to a point where AI can also fake both the location and actions that take place to levels that are indistinguishable from the real thing, then I think we’re in a spot of trouble.

or maybe we'll get back to living like 50-70 years ago, when there was minimal amount of video footage around. You trusted what you could see and touch.

It's not necessarily a bad thing.