We can't have nice things because of AI scrapers

Re:We can't have nice things because of AI scrapers by zimpenfish on Tuesday January 13, 2026 @06:10PM (#46609758)

I use iocaine[0] to generate a tarpit. Yesterday it served ~278k "pages" consisting of ~500MB of gibberish (and that's despite banning most AI scrapers in robots.txt.)

[0] https://iocaine.madhouse-project.org

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by chao- on Tuesday January 13, 2026 @06:41PM (#46610137)

Can't seem to access this.

It flashes some text briefly then gives me an 418 TEAPOT response. I wonder if it's because I'm on Linux?

EDIT: Begrudgingly checked Chrome, and it loads. I guess it doesn't like Firefox?

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by dpkirchner on Tuesday January 13, 2026 @07:48PM (#46610847)

Nor Safari on iOS.

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by zimpenfish on Wednesday January 14, 2026 @03:46AM (#46613793)

Works fine on my iOS Safari - maybe there's some extension that's tickling it just the wrong way?

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by dpkirchner on Wednesday January 14, 2026 @08:14AM (#46615682)

It still fails with all of my extensions disabled (wipr, privacy redirect). I just get a download dialog. I don't know what the HTTP status code is, however.

I found a flagged HN submission about it and it has just about the same result for me and for others. My first tap failed in a weird way (showed some text then redirected quickly to its git repo) and all subsequent taps trigger a download.

https://news.ycombinator.com/item?id=44538010

Given the replies here and in the older post, I suspect this tool is doing more harm to humans than to bots.

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by zephen on Tuesday January 13, 2026 @07:23PM (#46610633)

Doesn't work on my firefox either.

Friendly fire, I suppose.

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by godelski on Tuesday January 13, 2026 @07:40PM (#46610778)

Works on my Firefox. Mac and Linux

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by doublerabbit on Tuesday January 13, 2026 @06:40PM (#46610133)

Unfortunately and you kind of have to count this as the cost of the Internet. You've wasted 500Mb of bandwidth.

I've had colocation for eight years+. My monthly b/w cost is now around 20-30Gb a month given to scrapers where I was only be using 1-2Gb a month, years prior.

I pay for premium bandwidth (it's a thing) and only get 2TB of usable data. Do I go offline or let it continue?

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by zimpenfish on Wednesday January 14, 2026 @04:09AM (#46613937)

> You've wasted 500Mb of bandwidth.

Yep, it sucks, but on the positive side, I'm feeding 500Mb of garbage into them every day and that feels like enough of a small win for me.

> My monthly b/w cost is now around 20-30Gb a month given to scrapers [...] 1-2Gb a month

That definitely sucks.

> Do I go offline or let it continue?

Might be time to start blocking entire IP ranges and ASNs and see if that helps.

[ Parent | on HN ]

Re:We can't have nice things because of AI scrapers by zzzeek on Wednesday January 14, 2026 @12:22PM (#46618946)

i have no idea what this does because the site is rejecting my ordinary firefox browser with "Error code: 418 I'm a teapot". Even from a private browser.

If I hit it with Chrome, now I can see a site.

Seems pretty not ready for prime time as a lot of my viewers use Firefox

[ Parent | on HN ]