>companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have.

If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. Language models, for example, only produce plausibilities. You'll never be able to take even something in its context and guarantee it'll spit out something that's "100% correct" in response to a query or question on that information.

>And on top of that, companies also have to account for the potential of intentional trolling campaigns.

Yeah, they surely should "account" for them. I'm sure the individual responsible for the generation can be prosecuted under already existing laws. It's not really about safety at that point, and realistically about the corporation avoiding AGs raiding their office every week because someone incited a riot.

Ultimately, the cat's out of the bag in this case, and anyone who has amassed enough data and is motivated enough doesn't have to go to some AI startup to do any of this.

But perhaps the issue is not generative AI at that point, but humanity. Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.

> If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic.

The legal definition at play (at least when it comes to liability of companies under stuff like Section 230, GDPR, NetzDG or the planned DMA/DSA) is that a company makes reasonable best efforts/good faith to prevent harm. Everyone including lawmakers is aware that no system is perfect, but they do require that companies at least make an effort. Releasing a system capable of destabilizing nations (which deepfakes as a political weapon absolutely are) without any safeguards intentionally will get the hammer of the law brought upon them.

> Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.

For now, and for experts. Technology will evolve, post-production editing will be used to mask artifacts... and even if you have an expert that needs a day to verify inauthenticity, the damage will already be done. Rumors can spread in a matter of mere minutes.

Hell, just this week Turkey saw a potential deepfake porn, allegedly released by Russia, about Muharrem Ince damage his reputation enough to force him to resign from the election [1] - which turned out to be extremely close in the end. AI is a weapon of war, and it's in the hands of everyone.

[1] https://www.telegraph.co.uk/news/2023/05/14/turkey-deepfake-...

Section 230 is not about preventing "harm" or anything like that, the specific section just provides civil immunity for moderation/removal of content that is deemed "obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected."

You don't have a responsibility to remove such material under 230. But it gives you immunity to do so, as well as immunity from losing 230 protections for those that safeguard your overall immunity (as a service provider, and not a publisher.)

The purpose of the law is clear though: it recognizes the need of platforms to moderate problematic content and thus provides immunity when platforms moderate such content - effectively, it's an incentive for platforms to provide moderation.

Even the US has legal mandates to take down certain contents in any case - CSAM, IP violations (DMCA) and terrorist content come to my mind.

> Given how easy "hurt feelings" escalate into real-world violence

Well, not that I'm at all convinced, but that's more an argument for extreme soviet-style censorship rather than a minimalist "make sure AI has a political slant".