News for hackers. Stuff that matters.
Re:ChatGPT won't let you type until Cloudflare reads your Re... by mystraline on Sunday March 29, 2026 @06:53PM (#47568296)
This is a completely in-scope question.

How do we defend against your scraping, OpenAI?

I dont want any of my content scraped or seen by you all. Frankly, fuck you all for thinking my content is owned by you.

Re:ChatGPT won't let you type until Cloudflare reads your Re... by wilg on Sunday March 29, 2026 @07:49PM (#47568708)
Re:ChatGPT won't let you type until Cloudflare reads your Re... by tomalbrc on Sunday March 29, 2026 @09:10PM (#47569292)
Re:ChatGPT won't let you type until Cloudflare reads your Re... by wilg on Sunday March 29, 2026 @10:28PM (#47569747)
should be pretty easy to test and not rely on an anonymous source from a weird analytics company via business insider. are these bots actually from openai or are they just using their user agent? are they coming from openai ip ranges? etc. https://openai.com/gptbot.json
Re:ChatGPT won't let you type until Cloudflare reads your Re... by ImPostingOnHN on Monday March 30, 2026 @08:10AM (#47573234)
> should be pretty easy to test

I look forward to your results, whether or not they disprove the article.

Re:ChatGPT won't let you type until Cloudflare reads your Re... by stefanka on Monday March 30, 2026 @11:55AM (#47575882)
Are all of OpenAI’s ip ranges known?
Re:ChatGPT won't let you type until Cloudflare reads your Re... by stefanka on Monday March 30, 2026 @04:33AM (#47571892)
The article is from 2024. Is this still happening?
Re:ChatGPT won't let you type until Cloudflare reads your Re... by ImPostingOnHN on Monday March 30, 2026 @08:08AM (#47573225)
Do we have any evidence they started complying?

If not, we can conclude they did not, until such evidence shows up.

Re:ChatGPT won't let you type until Cloudflare reads your Re... by stefanka on Monday March 30, 2026 @11:24AM (#47575487)
I’m genuinely curious to know whether there was a change in behavior especially after OpenAI informed about how to prevent scraping (robot.txt, etc.).
Re:ChatGPT won't let you type until Cloudflare reads your Re... by ImPostingOnHN on Monday March 30, 2026 @11:29AM (#47575579)
I am as well. Like, is there any evidence of a change, or can we continue assuming nothing changed?
Re:ChatGPT won't let you type until Cloudflare reads your Re... by CableNinja on Monday March 30, 2026 @12:29AM (#47570398)
I use nginx conditionals and useragent checking, then respond with 418 or 410.

Probably too late now but my list needs updating

Re:ChatGPT won't let you type until Cloudflare reads your Re... by tedsanders on Monday March 30, 2026 @03:26AM (#47571451)
Re:ChatGPT won't let you type until Cloudflare reads your Re... by subscribed on Monday March 30, 2026 @10:31AM (#47574859)
- which one is to stop you from hammering small servers with hundreds simultaneous connections?

- why don't you just respect existing robots.txt that apply to you already?

- does every LLM scraper seriously think the onus to opt out from the EVERY SINGLE SCRAPER is on the webmasters/owners?

Re:ChatGPT won't let you type until Cloudflare reads your Re... by DrewADesign on Sunday April 12, 2026 @12:41AM (#47736214)
>* - why don't you just respect existing robots.txt that apply to you already?*

>* - does every LLM scraper seriously think the onus to opt out from the EVERY SINGLE SCRAPER is on the webmasters/owners?*

I see value in asking explicitly, but let’s be real here: you know the answers. They feel entitled to vacuum up everything visible, any way they feel like, for some ratio of profit to thinking they’re reshaping humanity for the better. The opt-outs are ceremonial procedures to convince people (including themselves,) that people are actually volunteering this data. They don’t give a shit about you, your creative output, you life’s work, your server performance and bandwidth, or anything else beyond stuffing the great data pig.