News for hackers. Stuff that matters.
Re:Gavin Newsom vetoes SB 1047 by alkonaut on Monday September 30, 2024 @04:13AM (#41694731)
The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated. Creating models that don't tag their output as such would be banned. So far nothing strange about the law. The obvious problem with the law is that you need to require even screenshotting an image AI and reposting it online without the made-with-ai metadata to be outlawed. And that would be an absolute mess to enforce, at least for images.

But most importantly: whatever we do in this space has to be made on the assumption that we can't really influence what "bad actors" do. Yes being responsible means leaving money on the table. So money has to be left on the table, for - erm - less responsible nations to pick up. That's just a fact.

Re:Gavin Newsom vetoes SB 1047 by worldsayshi on Monday September 30, 2024 @04:37AM (#41694869)
Any law that tries to categorize non-trustworthy content seems doomed to fail. We need to find better ways to communicate trustworthiness, not the other way around. (And I'm not sure adding more laws can help here.)
Re:Gavin Newsom vetoes SB 1047 by alkonaut on Monday September 30, 2024 @04:51AM (#41694946)
No I don't think technical means will work fully either. But the thing about these regulations is that you can basically cover the 99% case by just thinking about the 5 largest players in the field, be it regulation for social media, AI or whatever. It doesn't matter that the law has loopholes or that some players aren't affected at all. Regulation that helps somewhat in a large majority of cases is massive.
Re:Gavin Newsom vetoes SB 1047 by arder on Monday September 30, 2024 @06:13AM (#41695462)
I think the most acheivable way of having some verification of AI images is simply for the AI generators to store finger prints of every image they generate. That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you". There's already technology for that sort of thing in the world (content ID from youtube, CSAM detection etc.).

It's obviously not perfect, but could help and doesn't have the enormous side effects of trying to lock down all image generation.

Re:Gavin Newsom vetoes SB 1047 by Someone on Monday September 30, 2024 @06:54AM (#41695695)
> That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you".

Firstly, if you want to know an image isn’t generated, you’d have to go to every ‘whoever’ in the world, including companies that no longer exist.

Secondly, if you ask evil.com that question, you would have to trust them to answer honestly for both all images they generated and images they didn’t generate (claiming real pictures were generated by you can probably be career-ending for a politician)

This is worse than https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF: “Program testing can be used to show the presence of bugs, but never to show their absence!”. You can neither show an image is real nor that it is fake.

Re:Gavin Newsom vetoes SB 1047 by kortex on Monday September 30, 2024 @07:22AM (#41695894)
What's to stop someone from downloading an open source model, running it themselves, and either just not sharing the hashes, subtly corrupting the hash algo so that it gives a false negative, etc?

Also you need perceptual hashing (since one bitflip of the generated media alters the whole hash) which is squishy and not perfectly reliable to begin with.

Re:Gavin Newsom vetoes SB 1047 by alkonaut on Monday September 30, 2024 @01:40PM (#41699878)
Nothing. But that’s not the point. The point is that, to a rounding error, all output is made by a small number of models from a small number of easily regulated companies.

It’s never going to be possible to ensure all media is reliably tagged somehow. But if just half of media generated is identifiable as such that helps. Also helps avoid it in training new models, which could turn out useful.

Re:Gavin Newsom vetoes SB 1047 by anon291 on Monday September 30, 2024 @10:38AM (#41697627)
> Creating models that don't tag their output as such would be banned.

This is just silly. Anyone would be able to disable this tagging in an open model.

Re:Gavin Newsom vetoes SB 1047 by diggan on Monday September 30, 2024 @11:17AM (#41698157)
>> Creating models that don't tag their output as such would be banned.

> This is just silly. Anyone would be able to disable this tagging in an open model.

And we'd end up with people who thinks that any text that isn't tagged as "#MadeByLLM" as made by a human, which obviously wouldn't be great.

Re:Gavin Newsom vetoes SB 1047 by jerjerjer on Monday September 30, 2024 @01:58PM (#41700072)
> Anyone would be able to disable this tagging in an open model.

Metadata (I assume it's file metadata and not a watermark) can be removed from a final product (image, video, text) so open and closed models are equally affected.

Re:Gavin Newsom vetoes SB 1047 by drcode on Monday September 30, 2024 @03:19PM (#41700951)
turning the earth into paperclips is not gonna happen immediately, so we can safely ignore that risk
Re:Gavin Newsom vetoes SB 1047 by woah on Monday September 30, 2024 @06:44PM (#41702918)
> The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated.

Just make a law that makes it so that AI content has to be tagged if it is being used for fraud