Audio and video evidence isn't admissible because it's audio or video (and may be inadmissible nonetheless). It's admissible because someone testifies under oath that they have personal knowledge of its provenance. The burden is on the party introducing the evidence to show that it's reliable, and the question of whether or not it is indeed reliable is a factual one for a judge or jury to answer. It's not assumed to be "true" or to accurately reflect reality just because it's a purported photo, video, or audio recording.

It will be inadmissable because of the psychological effects. Imagine having someone claim you murdered someone. Also you happened to drive by that park where the person was murdered at the same time.

Now someone wants to frame you, perhaps because you are a celebrity or a person with money to extort from. So they claim that they saw you murder this person.

Normally, this wouldn't hold any water and no jury would convict you, we are not in the medieval ages anymore. There would be no evidence of you around the crime scene, because of course, you weren't there.

Now imagine that they are able to produce audio recordings of a conversation you had with the victim, screaming at them and saying "you are gonna pay for this". And then they also happen to have video evidence of a smart phone camera that happened to capture you stabbing the victim.

Now tell me, if the jury sees all this and then it gets dismissed, because of "deepfake" claims or whatever, how will it affect the outcome of this trial?

If you are truly innocent, this might still work out alright, unless you are not white. But imagine there is even the SLIGHTEST connection to the real world. It doesn't have to be completely fake. Maybe you had some fight with that victim earlier, or there are witnesses who testify under oath that you had several heated arguments with them, etc.

Deepfakes can change the entire outcome of trials, by biasing the jury and proceedings against you. That's why any audio and video evidence essentially needs to be rejected without a very very thorough forensic analysis of its authenticity. And even then you would still not know for sure.

Deepfakes will change the entire outcome of trials the same way photoshop changed the entire outcome of trials before it.

Also you should see the type of video evidence used in court. It’s grainy 5 FPS surveillance footage that doesn’t even show their face. Deepfakes are not necessary if you want to frame someone.

The jury wouldn’t see it if it got dismissed, typically by a Motion in Limine. Opposing counsel also gets to object to evidence before it’s shown to the jury during trial. If the trial judge allowed it and, on appeal, the appellate court decided it shouldn’t have been shown to the jury, there would be a new trial.

In your examples, it would most likely be a Mistrial scenario, the jury would be dismissed, and the prosecuting team would have to start over again from scratch later, assuming they aren't themselves in trouble for introducing faulty evidence.

I’d be really curious to hear about a case where prosecutorial misconduct was punished. Don’t they have something akin to the “qualified immunity” that the police enjoy?

Sci-fi author John Varley covered this in one of his novels.

In a society where anything digital can be manipulated, the veracity of a speech or event depends on a network of trust and people who witnessed the event in real life. If someone made a speech in front of hundreds of live, physically present spectators, these spectators can then verify the authenticity of any recording, and as long as there people in that group that you trust (directly, through your trusted network, or perhaps as journalists of good standing) you can trust that the recorded speech is what was said.

Conversely, a recording of a world leader saying they recommend Coca-Cola to prevent tooth decay wouldn't have any credibility at all if no one credible was actually there to witness it.

This doesn't work that well in practice:

- Humans can lie

- Human memory is too fallible to serve as this verification, particularly for every detail

- Small details with big impact can be changed in a verified speech without some people noticing

Of course if someone reliable records the actual speech, any copy can be checked against that. I'm not sure why human memory would be considered superior, as humans have poor memories and can lie.

How can you tell a true copy from a false copy? What makes one recording more true than the other, if you can only compare it against a reality which no longer exists (because it's in the past)?

I have a simple manufacturing tweak patent for the recorder devices on my table to do so.

What happens when there's a global pandemic and everyone is forced to stay home and not attend witness any of these speeches, but instead has to watch the broadcast over the internet and/or TV?

Then we just generate the speeches and let public figures do more work dealing with the pandemic, because it doesn't matter at that point.

Ideally anyways. Realistically, we probably just argue on the internet about which talks were real (in this case generated by the person pictured in them).

Cool idea for a sci-fi story.

Heh

So you'd need a bunch of GPT-3+ stuff to corroborate the deepfake, exciting! Where do you determine the "troll-line" in a trust network? You can build it over time I suppose? Though that kind of leads to similar echo-chamber stuff, right?

As an anecdote, I know real humans who do truly believe deepfaked political videos. They were genuinely surprised to know that Pelosi can talk like a normal human.

That’s a great point. Tons of people already don’t CARE about the facts, and consider fact-checkers to be just as biased as the statements they are checking. Just having 30%-90% of the population react to a deepfaked outrageous video will already be a huge damage even if 10% are stickler skeptics.

“A lie can go halfway around the world before the truth has a chance to get its pants on”

> and consider fact-checkers to be just as biased as the statements they are checking

Aren't they? I can come up with examples off the top of my head, let alone with a search. This is my favourite[1]:

> The fact-checker for the New Yorker who mistook a Marine veteran’s tattoo for a Nazi symbol has resigned, saying the “small mistake” has ruined her life.

"mistook" is being kind. Take a look at her Twitter feed[2] and tell me if you think she's less biased than the stories that would've passed across her desk, because her feed is incredibly partisan. Do you think she managed to remain impartial while she was still employed as a fact checker? I wonder if she intended or wanted to be impartial, let alone managed it. I also wonder how that kind of person comes to be part of a team of fact checkers, which brings into question their impartiality. How was she able to make this “small mistake” and no one else on that team caught it? Perhaps they lack editorial oversight like Snopes[3], the Snopes that employs people who are obviously biased:

> Of particular interest, when pressed about claims by the Daily Mail that at least one Snopes employee has actually run for political office and that this presents at the very least the appearance of potential bias in Snopes’ fact checks, David responded “It's pretty much a given that anyone who has ever run for (or held) a political office did so under some form of party affiliation and said something critical about their opponent(s) and/or other politicians at some point. Does that mean anyone who has ever run for office is manifestly unsuited to be associated with a fact-checking endeavor, in any capacity?”

I'd say it was unbelievable but it's really not[4].

> Snopes appears to be actively engaged in an effort to discredit and deplatform us.

Dropping one's scepticism because someone calls themselves an impartial fact checker strikes me as being in the same realm as following the orders of a complete stranger because they have a uniform and claim authority over you or a situation, i.e. unwise.

[1] https://nypost.com/2018/06/26/new-yorker-staffer-resigns-aft...

[2] https://twitter.com/chick_in_kiev

[3] https://www.forbes.com/sites/kalevleetaru/2016/12/22/the-dai...

[4] https://mailchi.mp/babylonbee/reparations-for-everyone-83635

Maybe there needs to be created a "Witness Coin" ledger, where events happened if enough people say they did.

What's needed is more cryptography. Build keys into sensors to sign media and use something like blockchain to broadcast the signature at time of capture / time when the device returns to the internet.

Won't be perfect, but it would be a hurdle. Also, for things like audio and video if you know the location on the planet then there are certain artifacts that are hard to fake with mere AI. The way sound bounces off walls in a room or shared environmental noise that other devices in the area would detect, like a plane flying by.

The problem with that is, the video can only be watched at it's true raw form. Any edits will need to be re-signed with "Edited by X". Now, X has to be trusted and not compromised (by money, bias or hacks).

If X is CNN, then half the population will not believe it. If X is Fox, the other half won't believe. Bottomline, the world will be truly living in virtual alternative universes and it doesn't even matter what truth is as long as everyone in that world believe that.

Watching and verifying are two separate procedures, if the edited piece has a link to the original source, the verification can pass to that.

This is actually a valid use case of blockchain, where each edit is signed by an entity in cascading fashion.

> Build keys into sensors to sign media and use something like blockchain to broadcast the signature at time of capture / time when the device returns to the internet.

That literally accomplishes nothing. It would be a matter of time before someone just removes the light sensor (CCD? or something) and has a custom made encoder that can write the input data as though the device actually records it live.

Yes, like I said, it's not perfect. There will still be deepfakes, but they won't be around every single nook and cranny. And once you ever invalidate a video, all other works by the same key are now suspect.

Imagine two court cases. In one, there is a video signed with a key from a phone that shows no signs of compromise. In the other, there is a video with no signatures at all. Which is more trustworthy? It's a continuum, not a binary.

Edit:

And also, we can scale trust. There should be more webs-of-trust in the world. I should be able to tap my phone next to my buddies and it should create a link between us. We should have the concept of trust online when we need it.

It's just a different world. Somehow we live with text being like that. Maybe we'll even learn to verify signatures from trusted sources one day.

With deepfakes, in general I agree but it is a bit like computer security. It may be no different from reality but it may still have marks of the network that generated it. Ways to detect it may come out slowly, like computer bugs, with some entities having incentives to research them heavily and not release them.

I think it’s because getting to a point where your error bound is that small with nature is incredibly difficult. GANs are a breakthrough but there are still lots of drawbacks and “tells” that allow you to identify a fake distribution. I actually don’t think we’ll get there for a really long time, and for all we know it might actually be impossible to replicate a natural distribution without gathering some untenable amount of data. Even if you gather that amount of data you often have no idea how correct your model actually is because you don’t have access to the natural distribution. You have to have a prior and picking that prior is a research area in and of itself. That’s why people say that. Your premise is really hard.

EDIT: just thought of something else, for videos and pictures increasing the resolution dramatically will make it harder to generate deep fakes, it might be interesting to see how much better cameras get and I’m sure that the amount of compute to fake a higher resolution image will be drastically higher. There are a lot of factors at play.

We do have methods to detect deepfakes and it often works very well (at least for any given generator, perhaps not multiple generators), but you're right, at some point deepfakes will be indistinguishable from real media. At that point I think the problem is largely outside the domain of computer science and we will need to start redefining "trust" looks like.

Anyone else find it mildly(/extremely) distressing that a large part of our industry is dedicated to creating technology that provides little-to-no value to society, but creates enormous opportunities for great societal damage?

We then release the tech to the wild without any proposed way to address the problems, instead saying “the problems this causes are outside the domain of CS, so I won’t bother considering them – besides, if I didn’t create these problems someone else would have”

I hope I didn't make it seem like I was pushing the solution onto others when I said that it will move outside the domain of computer science. My research is in the area of detecting deep fakes.

You're definitely right about how little we attend to the consequences of our advancements, especially in the field of AI/ML. This area is fraught with ethical and moral issues that have taken a backseat to the ideal of progress and I think we are making a mistake there.

If there were B and C that would mitigate the bad things in more powerful A, then fine

But otherwise delaying progress what if in the end a more powerful A will eventually make any B and C irrelevant? All you’re doing is kicking the can down the road maybe?

“Too sweet”

https://economicsbyfilm.net/blog/2015/11/25/is-ai-progress

Up until the last decades, photography was an art, and photos were generally manipulated, sometimes heavily , sometimes to remove people etc. This is not new, human societies have survived without video evidence since forever

And we see what happens when we do have video evidence because everyone has a camera in their pocket. For decades law enforcement version of events were taken as gospel. Now we have evidence that they do lie.

> photography was an art, and photos were generally manipulated, sometimes heavily , sometimes to remove people etc.

Except this was time consuming and couldn't be done automatically in an undetectable way.

> human societies have survived without video evidence since forever

Stupid argument, we've also survived war since forever, except now a war between nuclear powers would be devastating in a way war historically wasn't. It will absolutely not be the same as it was before photos and videos.

There's a scifi book where AGI arises from the ever escalating battle between spam bots and spam detectors, sometimes it feels like we're not too far away from that :)

I wonder how much more difficult it would be to deepfake a lightfield compared to a regular video.

Except of course if they also are able to use AI to generate hash collisions in everybcryptographically secure hash algorithm, but I highly doubt THAT’ll ever happen :-)

I'm not sure if we are there already. The Biden speech at the convention was very good, but the video was immediately claimed to be suspicious. The image had unusual softness around his mouth, compared to rest of the image. The speech was pre-recorded. So post processing could be responsible. But also deepfakes at this point are possible too. So who knows.

https://www.vice.com/en_us/article/7kzj8y/why-joe-bidens-vir...

security by obscurity - The companies detecting deepfakes will certainly not publish their methods/networks.

There are plenty of publicly published papers detailing detection of deep fakes.

And what about universities?

how would anyone know if their methods worked or they were just making results up then?