News for hackers. Stuff that matters.
Re:Paris prosecutors raid France offices of Elon Musk's X by miki123211 on Wednesday February 04, 2026 @12:31AM (#46881845)
This vindicates the pro-AI censorship crowd I guess.

It definitely makes it clear what is expected of AI companies. Your users aren't responsible for what they use your model for, you are, so you'd better make sure your model can't ever be used for anything nefarious. If you can't do that without keeping the model closed and verifying everyone's identities... well, that's good for your profits I guess.

Re:Paris prosecutors raid France offices of Elon Musk's X by culi on Wednesday February 04, 2026 @01:02AM (#46882016)
It's not really different from how we treat any other platform that can host CSAM. I guess the main difference is that it's being "made" instead of simply "distributed" here
Re:Paris prosecutors raid France offices of Elon Musk's X by toxik on Wednesday February 04, 2026 @02:20AM (#46882561)
Uh, let's distinguish between generated images, however revolting, and actual child sexual abuse.

The main problem with the image generators is that they are used to harass and smear people (and children...) Those were always illegal to do.

Re:Paris prosecutors raid France offices of Elon Musk's X by blipvert on Wednesday February 04, 2026 @04:13AM (#46883450)
Pro-tip: if you are actively assisting someone in doing illegal things then you are an accomplice.
Re:Paris prosecutors raid France offices of Elon Musk's X by krig on Wednesday February 04, 2026 @02:37AM (#46882679)
Those images are generated from a training set, and it is already well known and reported that those training sets contain _real_ CSAM, real violence, real abuse. That "generated" face of a child is based on real images of real children.
Re:Paris prosecutors raid France offices of Elon Musk's X by pavlov on Wednesday February 04, 2026 @02:59AM (#46882853)
Indeed, a Stanford study from a few years back showed that the image data sets used by essentially everybody contain CSAM.

Everybody else has teams building guardrails to mitigate this fundamental existential horror of these models. Musk fired all the safety people and decided to go all in on “adult” content.

Re:Paris prosecutors raid France offices of Elon Musk's X by exodust on Wednesday February 04, 2026 @10:03PM (#46895113)
What Stanford study?

If you mean the one titled "Generative ML and CSAM", I can't see any mention of your claim in that study. Care to explain?

https://fsi.stanford.edu/publication/generative-ml-and-csam-...

Re:Paris prosecutors raid France offices of Elon Musk's X by hdgvhicv on Wednesday February 04, 2026 @02:40AM (#46882703)
> Uh, let's distinguish between generated images, however revolting, and actual child sexual abuse.

If you want. In many countries the law doesn’t. If you don’t like the law your billion dollar company still has to follow it. At least in theory.

Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @04:42AM (#46883688)
> let's distinguish between generated images, however revolting, and actual child sexual abuse.

Can't because even before GenAI the "oh its generated in photoshop" or "they just look young" excuse was used successfully to allow a lot of people to walk free. the law was tightend in the early 2000s for precisely this reason

Re:Paris prosecutors raid France offices of Elon Musk's X by madeofpalk on Wednesday February 04, 2026 @03:25AM (#46883055)
Let’s take a step back and remove AI generation from the conversation for a moment.

Did X do enough to prevent its website being used to distribute illegal content - consensual sexual material of both adults and children?

Now reintroduce AI generation, where X plays a more active role in facilitating the creation of that illegal content.

Re:Paris prosecutors raid France offices of Elon Musk's X by muyuu on Wednesday February 04, 2026 @03:53AM (#46883289)
"Enough" can always be pushed into the impossible. That's why laws and regulations need to be more concrete than that.

There's essentially a push to end the remnants of the free speech Internet by making the medium responsible for the speech of its participants. Let's not pretend otherwise.

Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @04:45AM (#46883708)
The law is concrete on this.

In the UK, you must take "reasonable" steps to remove illegal content.

This normally means some basic detection (ie fingerprinting which is widely used from a collaborative database) or if a user is consistently uploading said stuff, banning them.

Allowing a service that you run to continue to generate said illegal content, even after you publicly admit that you know its wrong, is not reasonable.

Re:Paris prosecutors raid France offices of Elon Musk's X by muyuu on Wednesday February 04, 2026 @04:45AM (#46883712)
that doesn't sound concrete to me, at all
Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:00AM (#46884300)
No law is concrete. Murder is killing with intent to kill. What concrete test shows if someone intended to kill? They say you have intent to kill if a reasonable person would expect the actions you took would result in killing.
Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @05:53AM (#46884240)
Nothing in common law is "concrete", thats kinda the point of it.

Judges can evolve and interpret as they see fit, and this evolution is case law.

This is why in the US the supreme court can effectively change the law by issuing a binding ruling. (see 2nd amendment meaning no gun laws, rather than as written, or the recent racial profiling issues)

Re:Paris prosecutors raid France offices of Elon Musk's X by disgruntledphd2 on Wednesday February 04, 2026 @05:05AM (#46883866)
It's about as concrete as one gets in the UK/US/Anglosphere law tradition.
Re:Paris prosecutors raid France offices of Elon Musk's X by muyuu on Wednesday February 04, 2026 @05:09AM (#46883892)
if you can be sued for billions because some overbearing body, with a very different ideology to yours, can deem your moderation/censorship rules to be "unreasonable" then what you do is err on the side of caution and allow nearly nothing

this is not compatible with that line of business - perhaps one of the reasons nothing is done in Europe these days

Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:01AM (#46884309)
They advertised you could use the tool to undress people, that's pretty clearly on the unreasonable side of the line
Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @05:46AM (#46884191)
sigh

The vast majority of the EU is not common law, so "reasonable" in this instance is different.

What you describe already happens in the USA, that why MLB has that weird local TV blackout, why bad actors use copyright to take down content they don't like.

The reason why its so easy to do that is because companies must reasonably comply with copyright holder's requests.

Its the same with CSAM, distributing it doesn't have first amendment protection, knowingly distributing it is illegal. All reasonable steps should be taken to detect and remove CSAM from your systems to qualify for safe harbour.

Re:Paris prosecutors raid France offices of Elon Musk's X by muyuu on Wednesday February 04, 2026 @06:01AM (#46884305)
sigh indeed

> Its the same with CSAM, distributing it doesn't have first amendment protection, knowingly distributing it is illegal. All reasonable steps should be taken to detect and remove CSAM from your systems to qualify for safe harbour.

nice try, but nobody is distributing or hosting CSAM in the current conversation

people trying to trick a bot to post bikini pictures of preteens and blaming the platform for it is a ridiculous stretch to the concept of hosting CSAM, which really is a transparent attack to a perceived political opponent to push for a completely different model of the internet to the pre-existing one, a transition that is as obvious as is already advanced in Europe and most of the so-called Anglosphere

> The vast majority of the EU is not common law, so "reasonable" in this instance is different.

the vast majority of the EU is perhaps incompatible with any workable notion of free speech, so perhaps America will have to choose whether it's worth it to sanction them into submission, or cut them off at considerable economic loss

it's not a coincidence that next to nothing is built in Europe these days, the environment is one of fear and stifling regulation and if I were to actually release anything in either AI or social networks I'd do what most of my fellow Brits/Europoors do already, which is to either sell to America or flee this place before I get big enough to show up in the euro-borg's radar

Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @06:26AM (#46884511)
> nice try, but nobody is distributing or hosting CSAM in the current conversation

multiple agencies (Ofcom, irish police IWF, and what ever the french regulator is) have detected CSAM.

You may disagree with that statement, but bear in mind the definition of CSAM in the UK is "depiction of a child" which means that if its of a child or entirely generated is not relevant. This was to stop people claiming that massive cache of child porn they had was photoshoped.

in the USA CSAM is equally vaguely defined, but the case law is different.

> EU is perhaps incompatible with any workable notion of free speech

I mean the ECHR definition is fairly robust. But given that first amendment protection has effectively ended in the USA (the president is currently threatening to take a comedian to court for making jokes, you know, like the twitter bomb threat person in the UK) its a bit rich really. The USA is not the bastion of free speech it once was.

> either sell to America or flee this place before I get big enough to show up in the euro-borg's radar

Mate, as someone whos sold a startup to the USA, its not about regulations its about cold hard fucking cash. All major companies comply with EU regs, and its not hard. they just bitch about them so that the USA doesn't put in basic data protection laws, so they can continue to be monopolies.

Re:Paris prosecutors raid France offices of Elon Musk's X by disgruntledphd2 on Wednesday February 04, 2026 @08:56AM (#46885830)
> this is not compatible with that line of business - perhaps one of the reasons nothing is done in Europe these days

Except for 40% of all Big Tech products and a vast industrial network of companies, and the safe airplane building and decent financial services that don't take 3% of everything, then yeah, I guess nothing is done in Europe these days.

And wait, wasn't most of Google's AI stuff acquired from a European country?

Honestly, while Europe has a lot of problems, this notion that many US people have that literally nothing happens there is wildly off-base.

Re:Paris prosecutors raid France offices of Elon Musk's X by muyuu on Wednesday February 04, 2026 @09:17AM (#46886072)
Re:Paris prosecutors raid France offices of Elon Musk's X by disgruntledphd2 on Wednesday February 04, 2026 @10:55AM (#46887387)
Sigh.

Like, this is a function of fragmented capital markets rather than anything else. Ryanair would 100% have a market cap of 50bn+ if it had a US listing.

Anyway, market cap is a really really bad metric for basically anything. Like Walmart has a market cap of over 1tn now, do you think its business has materially changed since 2021 (when it was half that)?

Meta has basically doubled since then, and again, their business is basically the same as it was 5 years ago.

As another example, Stripe is valued at about 100bn, while both OpenAI and Anthropic are 3.5-5x that. Which ones would you rather put your money in?

If we were to look at income mobility the numbers look much more different: https://slowrevealgraphs.com/2024/01/13/income-mobility-acro...

(apologies I figured the OECD would have better data but this was the best I could find).

So, on average it would take a low-income person half the time to become upper income in Denmark versus the US. Is this a better metric? Certainly for a low income person with unlimited mobility options.

Re:Paris prosecutors raid France offices of Elon Musk's X by mnewme on Wednesday February 04, 2026 @03:24AM (#46883051)
This is the wrong take.

Yes they could have an uncensored model, but then they would need proper moderation and delete this kind of content instantly or ban users that produce it. Or don’t allow it in the first place.

It doesn’t matter how CSAM is produced, the only thing that matters is that is on the platform.

I am flabbergasted people even defend this

Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:02AM (#46884316)
It matters whether they attempt to block it or encourage it. Musk encouraged it, until legal pressure hit, then moved it behind a paywall so it's harder to see evidence.
Re:Paris prosecutors raid France offices of Elon Musk's X by mnewme on Wednesday February 04, 2026 @06:03AM (#46884331)
Exactly!
Re:Paris prosecutors raid France offices of Elon Musk's X by Jordan-117 on Wednesday February 04, 2026 @01:43AM (#46882329)
I could maybe see this argument if we were talking about raiding Stable Diffusion or Facebook or some other provider of local models. But the content at issue was generated not just by Twitter's AI model, but on their servers, integrated directly into their UI and hosted publicly on their platform. That makes them much more clearly culpable -- they're not just enabling this shit, they're creating it themselves on demand (and posting it directly to victims' public profiles).
Re:Paris prosecutors raid France offices of Elon Musk's X by disgruntledphd2 on Wednesday February 04, 2026 @05:08AM (#46883885)
And importantly, this is clearly published by Grok, rather than the user, so in this case (obviously this isn't the US) but if it was I'm not sure Section 230 would apply.
Re:Paris prosecutors raid France offices of Elon Musk's X by popalchemist on Wednesday February 04, 2026 @01:53AM (#46882395)
It's a bit of a leap to say that the model must be censored. SD and all the open image gen models are capable of all kinds of things, but nobody has gone after the open model trainers. They have gone after the companies making profits from providing services.
Re:Paris prosecutors raid France offices of Elon Musk's X by vintermann on Wednesday February 04, 2026 @03:14AM (#46882973)
So far, yes, but as far as I can tell their case against the AI giants aren't based on it being for-profit services in any way.
Re:Paris prosecutors raid France offices of Elon Musk's X by popalchemist on Wednesday February 04, 2026 @07:51PM (#46894172)
The for-profit part may or may not be a qualifier, but the architecture of a centralized service means they automatically become the scene of the crime -- either dissemination or storing of illegal material. Whereas if Stability creates a model, and others use their model locally, the relationship of Stability to the crime is ad-hoc. They aren't an accessory.
Re:Paris prosecutors raid France offices of Elon Musk's X by KaiserPro on Wednesday February 04, 2026 @04:49AM (#46883741)
Again its all about reasonable.

Firstly does the open model explicitly/tacitly allow CSAM generation?

Secondly, when the trainers are made aware of the problem, do they ignore it or attempt to put in place protections?

Thirdly, do they pull in data that is likely to allow that kind of content to be generated?

Fourthly, when they are told that this is happening, do they pull the model?

Fithly, do they charge for access/host the service and allow users to generate said content on their own servers?

Re:Paris prosecutors raid France offices of Elon Musk's X by themafia on Wednesday February 04, 2026 @01:13AM (#46882116)
Holding corporations accountable for their profit streams is "censorship?" I wish they'd stop passing models trained on internet conversations and hoarded data as fit for any purpose. The world does not need to boil oceans for hallucinating chat bots at this particular point in history.
Re:Paris prosecutors raid France offices of Elon Musk's X by vintermann on Wednesday February 04, 2026 @03:18AM (#46882998)
If it had been over profit, I would be all for it. I think that there are a ton of things which should be legal but not legal to make profit on.

But this is about hosting a model with allegedly insufficient safeguards against harassing and child-sexualizing images, isn't it?

Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:03AM (#46884328)
It's not about 'hosting a model'
Re:Paris prosecutors raid France offices of Elon Musk's X by vintermann on Wednesday February 04, 2026 @06:12AM (#46884402)
What do you mean? What is it about then?
Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:49AM (#46884718)
Publishing child porn
Re:Paris prosecutors raid France offices of Elon Musk's X by themafia on Wednesday February 04, 2026 @05:15PM (#46892619)
What would be censorship is if those same companies then brigaded forums and interfered with conversations and votes in an effort to try to hide their greed and criminality.

Not that this would _ever_ happen on Hacker News. :|

Re:Paris prosecutors raid France offices of Elon Musk's X by PeterStuer on Wednesday February 04, 2026 @03:12AM (#46882954)
Imagine the liabilities of camera producers. Better let those ballpoint manufacturers know they need to significantly up their legal insurance!
Re:Paris prosecutors raid France offices of Elon Musk's X by mnewme on Wednesday February 04, 2026 @03:26AM (#46883062)
That is not the same.

Correct comparison would be:

You provide a photo studio with an adjacent art gallery and allow people to shoot CSAM content there and then exhibit their work.

Re:Paris prosecutors raid France offices of Elon Musk's X by PeterStuer on Wednesday February 04, 2026 @03:36AM (#46883152)
Not knowing any better, and not having seen any of the alleged images, my default guess would be they used the exact same CSAM filtering pipeline already in place on X regardless of the origin of the submitted images.
Re:Paris prosecutors raid France offices of Elon Musk's X by mnewme on Wednesday February 04, 2026 @04:56AM (#46883792)
They obviously didn’t really implement anything as you can find that content or involuntary nudes of other people, which is also an invasion of privacy, super easily
Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:04AM (#46884334)
And the sign out front says "X-Ray camera photographs anyone naked — no age limits!"

And the camera is pointing out the window so you can use it on strangers walking by.

There is a point in law where you make something so easy to misuse that you become liable for the misuse.

In the USA they have "attractive nuisance", like building a kid's playground on top of a pit of snakes. That's so obviously a dumb idea that you become liable for the snake–bitten kids — you can't save yourself by arguing that you didn't give the kids permission to use the playground, that it's on private property, that the kids should have seen the snakes, or that it's legal to own snakes. No, you set up a situation where people were obviously going to get hurt and you become liable for the hurt.

Re:Paris prosecutors raid France offices of Elon Musk's X by brnt on Wednesday February 04, 2026 @03:37AM (#46883164)
If the camera reliably inserts racist filters and the ballpen would add hurtful words to whatever you write, indeed, let them up their legal insurance.
Re:Paris prosecutors raid France offices of Elon Musk's X by gordian-mind on Wednesday February 04, 2026 @05:19AM (#46883973)
This is not about AI but about censorship of a nonaligned social network. It's been a developing current in EU. They have basically been foaming at the mouth at the platform since it got bought.
Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @05:59AM (#46884292)
It's about a guy who thinks posting child porn on twitter is hilarious and that guy happens to own twitter.

If it was about blocking the social media they'd just block it, like they did with Russia Today, CUII-Liste Lina, or Pavel Durov.

Re:Paris prosecutors raid France offices of Elon Musk's X by mordnis on Wednesday February 04, 2026 @07:03AM (#46884840)
He said that child pornography is funny? Do you have a link by any chance?
Re:Paris prosecutors raid France offices of Elon Musk's X by gordian-mind on Wednesday February 04, 2026 @08:19PM (#46894372)
Although I despise it, I respect your right to lie through your teeths.
Re:Paris prosecutors raid France offices of Elon Musk's X by wtcactus on Wednesday February 04, 2026 @05:45AM (#46884176)
It's 2026. No common individual can be accountable for anything wrong they do. We must always find some way to blame some "corporation" or some "billionaire" or some ethnic group of people.

I wonder where all these people - and the French government - has been in the past 3 decades where kids did the same thing with Photoshop.

Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @06:02AM (#46884321)
You don't see any difference between Google Search and ChatGPT?
Re:Paris prosecutors raid France offices of Elon Musk's X by direwolf20 on Wednesday February 04, 2026 @05:56AM (#46884269)
It's not because it could generate CSAM. It's because when they found out it could generate CSAM, they didn't try to prevent that, they advertised it. Actual knowledge is a required compenent of many crimes.
Re:Paris prosecutors raid France offices of Elon Musk's X by bhelkey on Wednesday February 04, 2026 @06:21PM (#46893361)
> when they found out it could generate CSAM, they didn't try to prevent that, they advertised it.

Twitter publicly advertised it can create CSAM?

I have been off twitter for several years and I am open to being wrong here but that sounds unlikely.

Re:Paris prosecutors raid France offices of Elon Musk's X by code_for_monkey on Wednesday February 04, 2026 @09:17AM (#46886067)
I think having guardrails on your AI to not be able to produce this stuff is good actually. Also, Elon encourages this behavior socially through his posts so yeah he should face consequences.
Re:Paris prosecutors raid France offices of Elon Musk's X by keepamovin on Wednesday February 04, 2026 @09:45AM (#46886423)
There's no crowds or sides. It's all manufactured divisions because some of those who can't or don't want to create the technology are determined to control it. So they'll get you mad about what they need to, to justify actions that increase their control.

It's the same playbook that is used again and again. For war, civil liberties crackdowns, lockdowns, COVID, etc, etc: 0) I want (1); start playbook: A) Something bad is here, B) You need to feel X + Panic about it, C) We are solving it via (1). Because you reacted at B, you will support C. Problem, reaction, solution. Gives the playmakers the (1) they want.

We all know this is going on. But I guess we like knowing someone is pulling the strings. We like being led and maybe even manipulated because perhaps in the familiar system (which yields the undeniable goods of our current way of life), there is safety and stability? How else to explain.

Maybe the need to be entertained with drama is a hackable side effect of stable societies populated by people who evolved as warriors, hunters and survivors.

Re:Paris prosecutors raid France offices of Elon Musk's X by commandlinefan on Wednesday February 04, 2026 @12:12PM (#46888474)
Pretty disturbing to me how many people _on here_ are cheering for this. I thought that at least here of all places, there might be some nuanced discussion on "ok, I see why people are emotional about this topic in particular, but it's worth stepping back and putting emotions aside for a minute to see if this is actually reasonable overall..." but besides your comment, I'm not seeing much of that.
Re:Paris prosecutors raid France offices of Elon Musk's X by troyvit on Wednesday February 04, 2026 @12:21PM (#46888601)
There's pro-AI censorship and then there's pro-social media censorship. It was the X offices that were raided. X is a social media company. They would have been raided whether it was AI that created the CSAM or a bunch of X contractors generating it mechanical-turk style.

I think the HN crowd is more nuanced than you're giving them credit for: https://hn.algolia.com/?q=chat+control