Nothing exemplified this problem more than the recent Twitter takeover. On tiktok, fake news was running rampant, and I saw it happen in real time. At one point someone said that a Twitter employee banned Elon on his way out, while showing a screenshot of the “eIon musk” fake account: https://www.tiktok.com/t/ZTRx6G9mG/
There were also actual deepfakes of Elon’s voice that fooled me. I wish I saved it, since it’s highly relevant to this thread. But it only became clear it was a fake when it started saying ridiculous things. They even had a video of Elon talking, and they didn’t have to bother syncing his lips with the audio. Viral clips just need a veneer of authenticity to short circuit our monkey brains.
It wasn’t till I fell for that deepfake that I started realizing how much of a problem we might be facing. The next few presidential candidates might need to put out statements saying “X wasn’t true,” while the majority of people already believe X because they saw it happen live on tiktok.
I guess the counter (which I think you're pointing out) to that is to publish text to a known-trusted website rather than a video, but I wonder how many folks who get their "news" from TikTok would read it.
By "verified", I assume Twitter has verified that their identity on Twitter matches their real life identity, and I don't see a problem with anyone having an ability to claim that. What's wrong with Jane Random being verified that she _is_ Jane Random?
If people are basing their trust not on someone's identity and history, but on their celebrity (blue mark so far), that's where the problem is.
They then spoof being the real person, and verification then means absolutely nothing cause they are functionally identical to a bypasser.
By having verification be really hard to get, trolls have a lot harder time getting accounts and the problem is smaller in scope.
Having initial validation accessible is not the issue here.
It's like allowing verified emails to be changed to another address without reverifying.
I mean, doh.
He mentions things like "where/when did this happen", where were the people/things in the video at that time, which source did this arise from. Basically there's a whole web of context to any event, which is why the breaking news is the best place for this, but the long term viability is much harder.
He also mentions some NYT tech for timestamping their videos as they record them and to detect manipulation.
Mentioned on page 206, "The Perils and Opportunities of Artificial Intelligence" of the book.
https://c2pa.org/ seems to be the latest iteration of it.
> What is the Coalition for Content Provenance and Authenticity (C2PA)?
> The C2PA is a Joint Development Foundation project to collectively build an end-to-end open technical standard to provide publishers, creators, and consumers with opt-in, flexible ways to understand the authenticity and provenance of different types of media. C2PA opposes efforts to make content provenance measures mandatory.
What we can have is attestation and reputation, then provenance.
It starts with a container format that allows signatures of both the entire video with audio, audio only, and video only, along with reasonable segments (three seconds?) of each of these.
Excerpts of any of these can be transcluded into another such container. The editor can transform it in any way, the resulting display doesn't have to match the particulars of the excerpt (captioning, noise filtering, color correction) but the excerpt is itself included, or at least the signature.
It would be useful for some people to have cameras which do this at the moment of recording, along with GPS, cell location data, and so on, to get really bulletproof documentary footage.
I don't want that built into my phone, necessarily. As a container format and ffmpeg style tool, this would support anonymous attestation which can be demonstrated by the keyholder at any future time. As a built-in subpoena-able feature of my phone or mirrorless camera? Hmm, probably don't want that. If it snitches proactively, not a chance in hell.
Your comment in particular made me feel like it was worth posting!
We mustn't confuse attestation with reputation, nor even the combination of attestation and reputation with truth.
Attestation is nothing more than someone staking their reputation on the veracity of what they're sharing with the world.
If I'm being frank, I thought this was too obvious to need stressing. If the camera is pointed at David Copperfield, cryptography can't make the lens show you reality.
Attestation solves a lot of problems, but like my first sentence said, video integrity isn't a problem which can be solved anymore, if it ever was.
I’ve written a blog post that addresses many of problems, and your comment was one of the reasons I posted it! You can find it at https://news.ycombinator.com/item?id=33558566
Trusted timestamping can attest that a given piece of media was not created after a given instance in time.
Trusted hardware running trusted code, combined with remote attestation, can offer a high-degree of assurance that a given video clip is not a product of CGI/ML.
Digital signatures by the entity responsible for a video clip's capture/creation mitigates the creation of media not sanctioned by that entity. Subjects in a particular piece of media could additionally apply their digital signatures to further increase trust in its authenticity. (This is effectively a cryptographically enforced version of your third point.)
This isn't a silver bullet, and suffers all the pitfalls normally encountered in the implementation of cryptographic schemes.
The key difference is that digital signatures allow revocation-of-trust to be an effortless, automatic and delegatable process. As opposed to the current process that involves individual (and intrinsically limited) knowledge of who can be trusted. Frequently one doesn't even have provenance information for a media item, which is a prerequisite for making a trustworthiness judgment call.
The "hocus pocus" is literally the point that distinguishes this scheme from the plainly non-working system that we have today.
I'm not seeing anything in your description that couldn't also be said about using PKI to allow people to sign stuff they published. (And better: it's not broken by non-deceptive edits, like cropping or brightness adjustments)-- except that PKI already exists so we can't imagine that it would magically solve these problems. :) (and there is a long long history of people thinking varrious PKI ideas will magically solve varrious problems and then failing to solve them at all)
(I mean cameras that produced 'signed' output is also a thing that has previously existed too: https://blog.elcomsoft.com/2011/04/nikon-image-authenticatio... and was never particularly popular even with the target market)
Yes, I'm aware this leaves the door open for subversion of certification authorities by nation-states. It's still better than the nothing we have today.
> Anyway who is gonna build this?
If and when specialized imaging sensors become available, I don't see this being a huge lift. One wouldn't even need purpose-built cameras for all but the most high-assurance recording requirements. Remote attestation and trusted code execution is already being incorporated into most newer devices. Bringing it all together would mostly be a software problem at that point.
Of course, there would also need to be a value proposition to justify any kind of work towards this. I personally see this as an inevitability given the frightening pace of progress in the ML field, and the increasing public concern over things like deepfakes.
Thank you for your comments!
I don't see how it all comes together without a partnership between the tech companies that display media (Youtube, Facebook, Twitter, etc.) and the tech companies that provide hardware (Apple, Google, Samsung, etc.) Maybe what's needed is to go through a nonprofit like the Partnership on AI.
Google is an interesting target because they are the only company that's both a media displayer and a hardware provider. If Android devices get a "verified non-deepfake" capability before Apple devices get it, I could see that as being a significant leg up for Android actually. It encroaches on Apple's traditional branding of "we make products for important people". I wonder if the Titan M has capabilities which could be leveraged. https://android-developers.googleblog.com/2018/10/building-t...
With enough training data and modified network this can also be done.
1) Multiple camera angles. A simultaneous left / front / right view of a person speaking would be much harder to fake.
2) Higher-resolution video and high-quality audio are harder to fake, although this might not be a problem for entities with sufficient resources and motivation.
3) Attribution to a real person. While this raises problems for people who want to remain anonymous due to very reasonable fears of retaliation, it also makes it harder to push fakes out to social media.
Note many fake 'breaking news' videos are just stiched together old footage, no AI deepfake required. This has been noticeable across multiple platforms wrt Ukraine, for example:
https://www.bbc.com/news/60867414