News for hackers. Stuff that matters.
Re:Understanding and managing the impact of machine learning... by pmayrgundter on Thursday April 04, 2024 @05:34PM (#39936039)
I agree with the general idea of tagging content to help classify.

I'd given this some thought via MIME and ended up with a kind of BioNFT.. so named bc it uses NFTs piecewise, but tracking the creation events and agents types (bio, ai, etc) as part of the content lifecycle

https://twitter.com/PMayrgundter/status/1638016474483683328

Highlight..

What if:

  - devices sign source creations with a biosignature

  - editing tools sign input

  - media types include that, effectively saying:

    ai_edited(human_created(photo))
and do this under the experimental namespace in MIME:

  image/x.bio([email protected]/photo123).html

  image/x.adobe.photoai(http://x.bio([email protected]/photo123)).html
Re:Understanding and managing the impact of machine learning... by WJW on Thursday April 04, 2024 @06:35PM (#39936646)
Perhaps I am being too cynical, but how do you protect this scheme against hostile actors?

Bits don't have color after all (see https://ansuz.sooke.bc.ca/entry/23 if you don't get this reference) and it would be fairly trivial to manually alter such media types to anything you want. For example if someone posts a `ai_edited(human_created(photo))` online, it would be straightforward to take the pixels of the photo and re-publish them as a "new" image with only the `human_created(photo)` tags. You could also randomly start adding `ai_edited` tags to things you want discredited, etc.

Re:Understanding and managing the impact of machine learning... by pmayrgundter on Thursday April 04, 2024 @08:47PM (#39937577)
Ah no problem. Just a sketch..

Looking at C2PA, I'd now use "claim" to describe those tags. They're short and human readable but not self-contained. They'd have to be authenticated with tool signing to be very useful. As it also emphasizes, it's about establishing trust for a domain, eg can Samsung photo signing be trusted?

Eg for a normal photo, your device could sign it with.. and OTP coordinated with the manufacturer? Then include the signing metadata in the EXIF tags of the image. To validate, you send the image+signed_claims to manufacturer and they know the OTPs (?) and can verify the signing

Doesn't make it unforgeable, bc you could intercept the hardware otp/signing protocol.. but would raise the cost quite a bit. Maybe with frequent key resets you can raise this further

Something like that sounding more useful?

Re:Understanding and managing the impact of machine learning... by p1necone on Friday April 05, 2024 @02:17AM (#39939205)
The analogue hole makes this all pointless. Ultimately at some point down the chain there's regular camera sensors that can be replaced with anything you want.
Re:Understanding and managing the impact of machine learning... by dns_snek on Friday April 05, 2024 @03:48AM (#39939741)
Exactly. This won't stop any determined bad actor, if anything, it's just going to boost their credibility when they inevitably come up with a bypass that allows them to sign any piece of media as "authentic" (in a few days or weeks after a device implementing such a scheme is released).

What then? 99.9% of honest actors are paying with their privacy[1] and usability again, while remaining 0.1% of dishonest actors are using a bypass - which sounds a lot like piracy and various DRM schemes. We've been here before, and it never works.

[1] While this might not be an immediate goal, I think it's pretty obvious any such scheme would eventually be corrupted to encode a traceable device IDs into every photo and video taken with the device - to protect the children and stop CSAM, of course!

Re:Understanding and managing the impact of machine learning... by pmayrgundter on Friday April 05, 2024 @04:36PM (#39946899)
Hmm, how about watermarking the photo with some function of the OTP? (like bit steganography; invisible to normal human use)

Seems to preserve easy checking by device manufacturer, but not immediately clear to me how to replace the sensor in a way that gets a signed image.

And if it seems like a bit of trouble, I think it's still worth consideration by the manufacturer, as a feature for their images and support of a community of users who want to produce images that can be so validated

Re:Understanding and managing the impact of machine learning... by hiAndrewQuinn on Friday April 05, 2024 @07:07AM (#39940941)
A business of Internet-connected cameras which automatically hash and commit the hashes of photos they take, unaltered, would give that claim much more credence. I suppose this idea doesn't come up all that often because of the hardware changes involved, but I could see this being very handy for anyone who needs to verify that photos are real in a professional capacity.
Re:Understanding and managing the impact of machine learning... by HeatrayEnjoyer on Friday April 05, 2024 @09:22AM (#39942053)
>Doesn't make it unforgeable, bc you could intercept the hardware otp/signing protocol

If hardware driven attestation like that used by Xbox One is used, intercepting the protocol becomes impossible.

Preventing computing hardware from running unauthorized processes is already a largely solved problem.

Re:Understanding and managing the impact of machine learning... by Retr0id on Thursday April 04, 2024 @05:36PM (#39936061)
What you're describing here is basically what C2PA is
Re:Understanding and managing the impact of machine learning... by zerojames on Thursday April 04, 2024 @05:43PM (#39936122)
Reference, for context: https://c2pa.org/

And: the BBC just started using C2PA across some content. The BBC's R&D team talking about it: https://www.bbc.co.uk/rd/blog/2024-03-c2pa-verification-news...

Re:Understanding and managing the impact of machine learning... by rentnorove on Thursday April 04, 2024 @07:10PM (#39936964)
Related critique of the BBC's use of C2PA, and C2PA in general: https://www.hackerfactor.com/blog/index.php?/archives/1024-I...
Re:Understanding and managing the impact of machine learning... by dns_snek on Friday April 05, 2024 @11:33AM (#39943668)
That was an interesting rabbit hole of articles, thanks. From an earlier article: [1]

> At FotoForensics, I'm already seeing known fraud groups developing test pictures with C2PA metadata. (If C2PA was more widely adopted, I'm certain that some of these groups would deploy their forgeries right now.)

> To reiterate:

> * Without C2PA: Analysis tools can often identify forgeries, including altered metadata.

> * With C2PA: Identifying forgeries becomes much harder. You have to convince the audience that valid, verifiable, tamper-evident 'authentication and provenance' that uses a cryptographic signature, and was created with the backing of big tech companies like Adobe, Microsoft, Intel, etc., is wrong.

> Rather than eliminating or identifying fraud, C2PA enables a new type of fraud: forgeries that are authenticated by trust and associated with some of the biggest names on the tech landscape.

[1] https://www.hackerfactor.com/blog/index.php?/archives/1013-C...

Re:Understanding and managing the impact of machine learning... by pmayrgundter on Thursday April 04, 2024 @05:46PM (#39936164)
Thanks for the ref! Checking it out
Re:Understanding and managing the impact of machine learning... by yieldcrv on Thursday April 04, 2024 @05:57PM (#39936287)
C2PA would be better onchain so maybe you could do a proof of concept implementation of that
Re:Understanding and managing the impact of machine learning... by xeyownt on Friday April 05, 2024 @05:05AM (#39940267)
This C2PA looks flawed from the start. It just helps making the untrustworthy looks more trustworthy.
Re:Understanding and managing the impact of machine learning... by ctrw on Thursday April 04, 2024 @08:54PM (#39937611)
Trying to parse recursive urls seems like it will break everything everywhere.