Our justice system worked for a very long time in an environment where cell phones didn't exist, where cities weren't covered end-to-end in CCTV cameras, and our courts were able to do a decent job of protecting the innocent and putting away the bad guys. Distrust of video evidence only makes us revert back 30 years to when we weren't so monitored. It's something that we can survive. On the other hand, if we don't see an outright rejection of deepfakes in our courts, we're going to see it used as a political weapon. We can survive the former, but not the latter.
Unfortunately, I don't see it working out that way. Once courts and police get their hands on a new technology and it becomes sufficiently ingrained, it doesn't matter how little sense it makes. Drug dogs are bullshit, but they're still everywhere. [1] DNA evidence can be synthesized and planted, but it's the gold standard in terms of courtroom evidence. [2]
Given all of this, I don't have a lot of faith in our justice system getting deepfakes right, and that worries me a lot more than guilty people being presumed innocent.
[1]: https://www.washingtonpost.com/opinions/2019/02/05/supreme-c...
[2]: Good reason to avoid giving your DNA to 23andMe or to the state. Unfortunately, both my parents did 23andMe, so I'm screwed regardless. https://www.nytimes.com/2009/08/18/science/18dna.html
Actually, it probably is somewhat. I hate paying money but that my taxes go to useful things makes it bearable. I hate sucking up to a guy in a uniform but that he also arrests murderers makes it less infuriating.
But moreover, that's not the issue brought up originally, which was net damage to a group. The implication was that the justice system harmed minorities and as a group that's simply not true.
Also, the language used is trying to borrow outrage. Harmed minorities? No, harmed the poor. Many of whom were minorities. But there are rich racial minorities too.
Per the Innocence Project, 70% of the cases they have exonerated have been of a minority group, and often these cases involved underlying racial prejudice to railroad an innocent person into a long-term prison sentence or death.[2]
>As of November 2019, 367 people previously convicted of serious crimes in the United States had been exonerated by DNA testing since 1989, 21 of whom had been sentenced to death.[9] Almost all (99%) of the wrongful convictions were males,[19] with minority groups constituting approximately 70% (61% African American and 8% Latino).
1.https://www.sentencingproject.org/publications/un-report-on-...
2.https://en.wikipedia.org/wiki/Innocence_Project#Overturned_c...
If the problem is racism, in a country where sexism against women is said to be a major problem, why would there be a 99:1 preponderance of men? Are the racists also biased in favor of women and against men?
Are males a minority group? No, technically not, since sex split is about 50/50. Is the high preponderance of men ending up in prison concerning? Yes. Frankly our high incarceration rates for non-violent crimes is extremely concerning. Does this somehow invalidate the problems brought up around female equality? Absolutely not. I also believe there is probably additional negative bias towards African-American males because of racist tropes.
Also the 99% figure was from the Innocence Project, Google brings up Federal Bureau of Prisons stat suggesting active incarceration is closer to 93% male, 7% female. Still concerning.
Hasn't anyone told you that you learn more with questions than assertions?
No. I'm saying that the language used is throwing out the baby with the bathwater. Regardless of all the stuff you mentioned, which is true, the system is still a benefit.
You should not go out of your way to further weaken trust in the system that has benefitted those minorities you mention more than it has hurt them. Especially as you are presumably not those groups, you should be careful not to wreck what they have. (You may not be aware, but white 'progressives' often speak for people of color. Their messages sound like ones of support initially, but because these people are often merely social signaling the rhetoric can often prove harmful to people who have to live with it.)
> [...] all disproportionately impact minority groups compared to Caucasians
Some minorities, yes. Others, no. Deeply troubling to the white v black narrative is that Nigerian immigrants often do very well in the USA, even when Americans don't know if they're american-descendants-of-slavery or not.
But yes, ADoS do have it rough. If you want to support someone though, vague "minorities" is not how you do it.
> Per the Innocence Project, 70% of the cases
You can't use that stat in that way, presumably they picked the most egregious cases which would be the poorest, etc.
>You can't use that stat in that way, presumably they picked the most egregious cases which would be the poorest, etc.
No, you cannot just ignore ingrained racism within the judicial system from day one as simply being a class issue. You can read more from the National Registry of Exonerations about racial bias in exonerated cases[1].
1. http://www.law.umich.edu/special/exoneration/Documents/Race_...
Just because some abuse them and have no proper training doesn't mean they are fake. You clearly have never seen a police dog work. They are able to find dead people, drugs, etc. that is at the bottom of the sea or follow the path the target took while inside a car. What is BS is the "police" and justice system in the US.
Yes, dogs are capable of sniffing out people, animals, and substances. This has been made use of for centuries.
Dogs are also capable of "hitting" on the trunk of a car, just because they guess that their master wants to look inside it. Plenty of evidence that this happens all the time.
Which undermines the right protecting against unreasonable search and seizure. All you have to do is be the wrong color, or driving in a "suspicious" way (which means whatever police want it to mean), and they can bring a K9 by, the dog will dutifully point at the trunk, and the cops get their search.
It's no different from the "I smelled pot" routine: there aren't any penalties to the police for doing an "I smelled pot" search and not finding cannabis, and there are no consequences for a K9 'hitting' on a trunk which turns out to have nothing more interesting than a gym bag.
The prosecution projected a film with audio [cutting edge technology at the time] demonstrating the crimes in question [bribery and related charges].
Lamont ['The Shadow'] was able to hear the audio and read lips at the same time and figure out that what was heard was not was what said. "Believe half of what you see, and nothing you hear" was his takeaway.
As the story goes, that's what happened. The film shown at court was dubbed for audio by a hired guy who did great vocal impersonations. The accused was innocent.
Short story long: deep fakes aren't a new idea. 100 years and counting. Same trick, different tools.
This kind of "verification" doesn't work. You're trying to have the device authenticate the image. That implies a root of trust that goes through the device manufacturer, which state actors will immediately compromise, making the signatures untrustworthy regardless. Then people will find vulnerabilities in the devices themselves, or convincing ways to have the device take a picture of a doctored photograph instead of a picture of the world, extending the ability to forge signatures to the general public.
The resources needed to forge a signature aren't going to be much different than the resources needed to create a deepfake to begin with. The creator gets to choose which device to use and can choose based on which device they find a vulnerability in. So it's more likely to be a liability than a benefit because it lends false credence to the authenticity of fakes.
For it to work you would need all devices to be secure against all attackers. It's not realistic.
Yes, if a state actor wants to create evidence to frame you, and they didn't care about cost or returns, they could do so. But they could always do so. The fact is most of us are never going to get framed by a nation state.
The notion that you're either secure against all possible attacks or nothing is absurd.
> Yes, if a state actor wants to create evidence to frame you, and they didn't care about cost or returns, they could do so. But they could always do so. The fact is most of us are never going to get framed by a nation state.
This is a major problem for elections. Nation states absolutely will doctor video of adversarial political candidates and for other propaganda purposes.
> The notion that you're either secure against all possible attacks or nothing is absurd.
But that's how signatures work. You're trusting every device in the world to sign the output of its camera. If the attacker can compromise any device, they can produce signed forgeries.
Again, nation states have always been able to doctor video of adversarial political candidates. Lookup pictures where the KGB made someone disappear.
Yes, if an attacker can compromise any device they can produce a signed forgery. But that's a really big if. There are a lot of potential attackers who can't compromise any and all devices, and signatures protect against them.
How? If you have a public/private key pair and then do some kind of multisig process wherein the manufacturer can sign the photo, the OS can sign the photo, and then a user supplied signature could also sign the photo. You're not going to be able to fake 3 cryptographic signatures without 3 private keys. It would be extremely difficult to fake all of that. It would take a concerted, concentrated effort and would be extremely rare in practice. It would certainly improve the situation.
Or, you'd adapt a hardware wallet to do the same. Either way, there'd be two devices from two manufacturers, communicating over a transparent protocol.
This image can now be injected into the news cycle and be taken seriously.
Keeping a secret on the camera is more or less the "jailbreak problem", aka being able to make a dvd player or console that only does the things that the manufacturer wants.
I will be pretty sad if this problem is actually "solved" effectively because it portends a future where manufacturers can completely curtail user freedom. But...
If the key on the device is compromised by reverse engineering or because the manufacturer is compromised then the whole root of trust is gone.
Attackers can choose the weakest brand of camera.
You now have some doubt about any image that has been "verified" because it could have come from a compromised camera secret.
All the rest of the stuff, the blockchain, etc depends on the root of trust in the camera.
You also now have debates around camera country of origin because there is no near future world where governments can't lean on local manufacturers. Finally, doubt can be thrown on images retro-actively if we later learn that a camera we thought was secure, has vulnerabilities.
All of this would be optional, of course. People can always edit out metadata. But it would lend credence and authority to a photograph or video. Which would carry more weight in a court of law, for example, where this stuff really matters.
I think the disconnect might be that you are working from a threat model where someone else is trying to tamper with an image from your camera. I agree you could make that arbitrarily difficult. But I am primarily concerned with the threat where someone is tampering with their own camera to fake evidence. This possibility makes it harder for you to prove that your own image is legitimate.
Your reasoning implies that one or two hacked cameras are not a problem. To my way of thinking, the possibility that "at least 1 hacked camera exists" or "any camera could possibly be hacked" are enough to throw doubt on any "verified" images that someone tries to present in a high stakes context such as national security or the courtroom. We're back to debating provenance.
It would require a larger degree of premeditation and effort to convincingly pull off a deep fake in that world.
It's not a technicality if the video can't be distinguished from perfect nonsense. Just as you don't let someone with a documented history of hallucinations testify on their own.
I imagine deepfakes will become way easier in the near future because there are legitimate commercial reasons to make them easy and accessible (for example digital art assets). I don't see the same happening for cracking devices, which is more a standard security cat and mouse game between hackers and device manufacturers.
If someone presents altered footage, you can show that the checksum of the original was uploaded earlier.
This of course only covers certain cases. If the deepfake is crafted from scratch or from a camera that doesn't upload its checksums, and the incentive for the creation of the deepfake arises before the supposed time the deepfake takes place, then this method won't be of use.
And subject to attack. Don't want to be recorded? Jam wireless so they can't have anyone sign their videos, then claim they're fake.
Also doesn't work for state actors. A country can compromise three separate signing authorities.
And you can just upload it to ten different authorities in ten different jurisdictions, or even a hundred if you're paranoid. States are powerful but they're not _that_ powerful. It's the same principle that makes Tor work.
I agree jamming wireless could be an issue, but only for prosecution. In the case of proving innocence it's a moot point because this checksum system only covers after-the-fact fabrication anyway.
They'd also have logged the IP, which combined with a time will probably significantly narrow down the individual in question if you can also access telco records. Especially if inadvertently more information is also documented (such as a user agent).
> It's the same principle that makes Tor work.
Tor doesn't work for this kind of problem, unless you are significantly more technically savvy (and even then...) then the average smartphone user.
Scaling the image down seems like one of the best tricks if you can get away with it. Another cheap one would be shifting the image 4 pixels over to break up the DCT artifact grid.
With a good forgery it will be. Even then, the fact that it's noise means that you won't ever be sure about this.
Are you saying that by law devices should cryptographically sign pictures?
Just saying "we need to sign photos" isn't enough and simply naïve.
The reality is that for most of those tens of thousands of years justice systems were mostly a sham where the focal point wasn't actually determining innocence but rather the appearance of justice and pleasing (some) people.
That future sounds a lot like the past where we couldn't trust testimony and audiovisual evidence was practically nonexistent.
I think the danger is in trying to come up with an abstract, general solution for it. I don't think one exists.
I think the only thing we can do is deal with things on a case by case basis in these situations.
I remember that TruePic was especially awful, because their app fell for me just taking a photo of a printed out image. Plus their website overflows with Blockchain and Crypto buzzwords
Not to mention adversarial attacks will probably always exist, so there will always be a way to fool these systems.
This.
I have been in a jury, and I wouldn’t be surprised if image based evidence won’t always be discounted by defense attorneys. It probably is already starting for the high paid ones (e.g. the notion itself will protect the rich.)
It brings up an interesting future where you can’t trust any witness testimony (because people do misremember) or any video, image, or audio evidence due to this technology.
The only societal workaround will be to implant cryptographic keys inside of bodies and monitor all interactions on some blockchain /s?