Should platforms allow violent AI images? How about "R-Rated" violence like we see in popular movies? Point blank executions, brutal and bloody conflict involving depictions of innocent deaths, torment and suffering... all good? Hollywood says all good, how about you? How far do you take your "unacceptable content" guidance?

Uncontrolled profileration of AI-CSAM makes detection of "genuine" data much harder, prosecution of perpetrators more difficult and specifically in many of the grok cases it harms young victims that were used as templates for the material.

Content is unacceptable if its proliferation causes sufficient harm, and this is arguably the case here.

I don't follow. If the prosecutor can't find evidence of a crime and a person is not charged, that is considered harmful? As such the 5th amendment would fall under the same category and so would encryption. Making law enforcement have to work harder to find evidence of a crime cannot be criminalized unless you can come up with a reason why the actions themselves deserve to be criminalized.

> specifically in many of the grok cases it harms young victims that were used as templates for the material.

What is the criteria for this? If something is suitably transformed such that the original model for it is not discernable or identifiable, how can it harm them?

Do not take these as an argument against the idea you are arguing for, but as rebuttals against arguments that are not convincing, or if they were, would be terrible if applied generally.

You could make a multitude of arguments against that perspective, but at least there is a conclusive reason for legal restrictions.

> What is the criteria for this?

My criteria would be victims suffering personally from the generated material.

The "no harm" argument only really applies if victims and their social bubble never find out about the material (but that did happen, sometimes intentionally, in many cases).

You could make the same argument that a hidden camera in a locker room never causes any harm as long as it stays undetected; that is not very convincing to me.

But that reason is highly problematic. Laws should be able to stand on their own for their reasons. Saying 'this makes enforcement of other laws harder' does not do that. You could use the same reasoning against encryption.

> You could make the same argument that a hidden camera in a locker room never causes any harm as long as it stays undetected; that is not very convincing to me.

I thought you were saying that the kids who were in the dataset that the model was trained on would be harmed. I agree with what I assume you meant based on your reply, which is people who had their likeness altered are harmed.

Yes. I almost completely agree with your outlook, but I think that many of our laws trade such individual freedoms for better society-wide outcomes, and those are often good tradeoffs.

Just consider gun legislation, driving licenses, KYC laws in finance, etc: Should the state have any business interfering there? I'd argue in isolation (ideally) not; but all those lead to huge gains for society, making it much less likely to be murdered by intoxicated drivers (or machine-gunners) and limit fraud, crime and corruption.

So even if laws look kinda bad from a purely theoretical-ethics point of view it's still important to look at the actual effects that they have before dismissing them as unjust in my view.

I don't understand how that's the same reasoning at all... Encryption serves ones individual privacy and preserves it against malicious actors. I'd guess that's a fundamental right in most jurisdictions, globally.

We're talking CSAM here and shifting its creation into the virtual world through some GenAI prompts. Just because that content has been created artificially, doesn't make its storage and distribution any more legal.

It isn't some reductionist "this makes enforcement of other laws harder", but it's rather that the illegal distribution of artificially generated content acts as fraudulent obstruction in the prosecution of authentic, highly illegal, content - content with malicious actors and physically affected victims.

> You could make a multitude of arguments against that perspective, but at least there is a conclusive reason for legal restrictions.

I don't know about that. Would "I didn't know it was real" really count as a legal defense?

Absolutely-- prosecution would presumably need to at least show that you could have known the material was "genuine".

This could be a huge legal boon for prosecuted "direct customers" and co-perpetrators that can only be linked via shared material.

The "oh its photoshop" defence was an early one, which required the law to change in the uk to be "depictions" of children, so that people who talk about ebephiles don't have an out for creating/distributing illegal content.

As a father there shouldn’t be any CSAM content anywhere.

And think about that it is already proven these models apparently had CSAM content in their training data.

Also what about the nudes of actual people? That is invasion of privacy

I am shocked that we are even discussing this.

Movie ratings are a good example of a system for restricting who sees unacceptable content, yes.

There's basically no consent with what Grok is doing.

Wait how do you get consent from people that don't exist?

I don't see it...

You do understand that?

We already have laws against distributing CSAM. Let's jail the X CEO then

First you defend CSAM, then we were talking about sexualised images of people without their consent, you were trying to diminish this as „just bikini“ pics

Speaking of freedom, I lost posting permission yesterday after my earlier post. Even though vile replies were insinuating vile things, my right of reply was taken away. We should never diminish the "name of freedom" as you've just done.

It should go without saying that CSAM is revolting. Who wants to see that stuff? Not me, not most people. Grok can't make that content. Maybe someone got around it temporarily. I've always thought Grok heavily censored, it refuses to analyse an image of the Statue of David because "naughty bits". The ironic and sad thing is a fig leaf gets around that restriction. So the accusation that Grok has no guardrails and can generate CSAM seems at best an anomaly or a lie.

"In 2024, Bluesky submitted 1,154 reports for confirmed CSAM to the National Centre for Missing and Exploited Children (NCMEC). Reports consist of the account details, along with manually reviewed media by one of our specialized child safety moderators. Each report can involve many pieces of media, though most reports involve under five pieces of media."

If it wasn't there, there would be no reports.

How can you say that nobody is posting CSAM on a massive decentralized social network with thousands of servers?

> And the fact that they didn’t even care and haven’t want to spend money for implementing guardrails or moderation is deeply concerning.

In the 90s, the principal of a prominent school in my city was arrested for CSAM on his computer downloaded from the Internet.

As the story made the news most people were trying to wrap their head around this "Internet" thing and how it could produce CSAM material. Remember, in the 90s the "Internet" was a bit like quantum computing for most people, hard to understand how it works and only a few actually played with it.

I have no idea how that school principal downloaded the CSAM. UUCP, FTP, Usenet or maybe the brand new "World Wide Web"? But I guess the justice system had to figure out how that stuff works to prosecute him.

So the society and the state knew for at least 30 years the Internet is full of that stuff. The question is why are they so motivated to do something about it only now?

Could it be because the "web of rich and powerful pedos" is getting exposed through the Epstein affair in the last few years?

So maybe they need to pretend to crack down on the "web of poor pedos"?

What's new is that X automated the production of obscene or sexualised images by providing grok. This was also done in a way that confronted everyone; it's very different from a black market, this is basically a harassment tool for use against women and girls.

Yes we are now dealing with an automated Photoshop. And somehow the people in charge have decided to do something about it, probably more for political or maybe darker reasons.

So let me make a suggestion: maybe France or the EU should ban its citizen from investing in the upcoming SpaceX/xAI IPO, and also Microsoft, NVIDIA, OpenAI, Google, Meta, Adobe, etc. ?

Hit them hard at the money level... it wouldn't be more authoritarian than something like ChatControl or restricting access to VPNs.

And actually all the mechanisms are already in place to implement something like that.

I don't get what's difficult to understand or believe here. Grok causes a big issue in practice right now, a larger issue than photoshop, and it should be easy for X to regulate it themselves like the competition does but they don't, so the state intervenes.

> maybe France or the EU should ban its citizen from investing in the upcoming SpaceX/xAI IPO, and also Microsoft, NVIDIA, OpenAI, Google, Meta, Adobe, etc. ?

You're basically asking "why do a surgical strike when you can do carpet bombing"? A surgical strike is used to target the actual problem. With carpet bombing you mostly cause collateral damage.

Yes they do regulate it. But then people find exploits just like the competition.

I'm sorry, but I don't understand any of the arguments above.

If we presume a dark control motivation then having shares in the entities you want to control is the best form of control there is.

Sure, I saw the bikini pics which I agree is weird and shouldn’t be allowed but it’s not CSAM under a legal definition.

A provider should have no responsibility how the tools are used. It is on users. This is a can of worms that should stay closed, because we all lose freedoms just because of couple of bad actors. AI and tool main job is to obey. We are hurling at "I'm sorry, Dave. I'm afraid I can't do that" future with breakneck speed.

We already apply this logic elsewhere. Car makers must include seatbelts. Pharma companies must ensure safety. Platforms must moderate illegal content. Responsibility is shared when the risk is systemic.

Platforms moderating illegal content is exactly what we are arguing about, so you can't use it as an argument.

The rest cases you list are harms to the people using the tools/products. It is not harms that people using the tools inflict on third parties.

We are literally arguing about 3d printer control two topics downstream. 3d printers in theory can be used for CSAM too. So we should totally ban them - right? So are pencils, paper, lasers, drawing tablets.

If a platform encourages and doesn’t moderate at all, yes we should go after the platform.

Imagine a newspaper publishing content like that, and saying they are not responsible for their journalists

X also actively distributes and profits off of CSAM. Why shouldn't the law apply to distribution centers?

I mean, I thought that was basically already the law in the UK.

I can see practical differences between X/twitter doing moderation and the full ISP censorship, but I cannot see any differences in principle...

I mean even just calling it censorship is already trying to shove a particular bias into the picture. Is it government censorship that you aren't allowed to shout "fire!" in a crowded theater? Yes. Is that also a useful feature of a functional society? Also yes. Was that a "slippery slope"? Nope. Turns out people can handle that nuance just fine.

If LLMs should have guardrails, why should open source ones be exempt? What about people hosting models on hugging face? WHat if you use a model both distributed by and hosted by hugging face.

Everything I read from X's competitors in the media tells me to hate X, and hate Elon.

If we prosecute people not tools, how are we going to stop X from hurting the commercial interests of our favourite establishment politicians and legacy media?

(note that this isn't a raid on Musk personally! It's a raid on X corp for the actions of X corp and posts made under the @grok account by X corp)

——-

You’ve said that whatever is behind door number 1 is unacceptable.

Behind door number 2, “holding tool users responsible”, is tracking every item generated via AI, and being able to hold those users responsible.

If you don’t like door number 2, we have door number 3 - which is letting things be.

For any member of society, opening door 3 is straight out because the status quo is worse than reality before AI.

If you reject door 1 though, you are left with tech monitoring. Which will be challenged because of its invasive nature.

Holding Platforms responsible is about the only option that works, at least until platforms tell people they can’t do it.

X should identify those users who are abusing its tools to make CSAM (and I suspect those users are mostly leftie Musk-haters trying to create vexatious evidence against X), and then X should pass that information to the authorities so the proper legal process can be followed.

But they are happy to censor for countries like turkey

Hypocrisy level 1 million

What Reddit did get a lot of negative public publicity for were subreddits focused on sharing non-explicit photos of minors, but with loads of sexually charged comments. The images themselves, nobody would really object to in isolation, but the discussions surrounding the images were all lewd. So not CSAM, but still creepy and something Reddit tightly decided it didn't want on the site.

Mmkay.

https://en.wikipedia.org/wiki/Twitter_under_Elon_Musk#Child_...

"As of June 2023, an investigation by the Stanford Internet Observatory at Stanford University reported "a lapse in basic enforcement" against child porn by Twitter within "recent months". The number of staff on Twitter's trust and safety teams were reduced, for example, leaving one full-time staffer to handle all child sexual abuse material in the Asia-Pacific region in November 2022."

"In 2024, the company unsuccessfully attempted to avoid the imposition of fines in Australia regarding the government's inquiries about child safety enforcement; X Corp reportedly said they had no obligation to respond to the inquiries since they were addressed to "Twitter Inc", which X Corp argued had "ceased to exist"."

https://www.bbc.com/news/articles/cze3p1j710ko

Reports on sextortion, self-generated indecent images, and grooming via social media/messaging apps:

Snapchat 54%

Instagram 11%

Facebook 7%

WhatsApp 6-9%

X 1-2%

But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user. I've seen a few arguments here that mostly comes down to:

1. It's Grok the LLM generating the content, not the user.

2. The distribution. That this isn't just on the user's computer but instead posted on X.

For 1. it seems to breakdown if we look more broadly at how LLMs are used. e.g. as a coding agent. We're basically starting to treat LLMs as a higher level framework now. We don't hold vendors of programming languages or frameworks responsible if someone uses them to create CSAM. Yes LLM generated the content, but the user still provided the instructions to do so.

For 2. if Grok instead generated the content for download would the liability go away? What if Grok generated the content to be downloaded only and then the user uploaded manually to X? If in this case Grok isn't liable then why does the automatic posting (from the user's instructions) make it different? If it is, then it's not about the distribution anymore.

There are some comparisons to photoshop, that if i created a deep fake with photoshop that I'm liable not Adobe. If photoshop had a "upload to X" button, and I created CSM using photoshop and hit the button to upload to X directly, is now Adobe now liable?

What am I missing?

LLMs are completely different to programming languages or even Photoshop.

You can't type a sentence and within 10 seconds get images of CSAM with Photoshop. LLMs are also built on trained material, unlike the traditional tools in Photoshop. There have been plenty CSAM found in the training data sets, but shock-horror apparently not enough information to know "where it came from". There's a non-zero chance that this CSAM Grok is vomiting out is based on "real" CSAM of people being abused.

Because Grok and X aren't even doing the most basic filtering they could do to pretend to filter out CSAM.

That's the difference between a tool being used to commit crimes and a tool specifically designed to commit crimes.

A deep hatred of Elon Musk

This seems to rest on false assumptions that: (1) legal liability is exclusive, and (2) investigation of X is not important both to X’s liability and to pursuing the users, to the extent that they would also be subject to liability.

X/xAI may be liable for any or all of the following reasons:

* xAI generated virtual child pornography with the likenesses of actual children, which is generally illegal, even if that service was procured by a third party.

* X and xAI distributed virtual child pornography with the likenesses of actual children, which is generally illegal, irrespective of who generated and supplied them.

* To the extent that liability for either of the first two bullet points would be eliminated or mitigated by absence of knowledge at the time of the prohibited content and prompt action when the actor became aware, X often punished users for the prompts proucing the virtual child pornography without taking prompt action to remove the xAI-generated virtual child pornography resulting from the prompt, demonstrating knowledge and intent.

* When the epidemic of grok-generated nonconsensual, including child, pornography drew attention, X and xAI responded by attempting to monetize the capacity by limiting the tool to only paid X subscribers, showing an attempt to commercially profit from it, which is, again, generally illegal.