News for hackers. Stuff that matters.

A comment on I protect my forgejo instance from AI Web Crawlers. Parent comment

Re:I protect my forgejo instance from AI Web Crawlers by bingo-bongo on Monday December 22, 2025 @05:49AM (#46353057)
AI companies scrape to:

- have data to train on

- update the data more or less continuously

- answer queries from users on the fly

With a lot of AI companies, that generates a lot of scraping. Also, some of them behave terribly when scraping or is just bad at it.

[ Parent | on HN ]
Re:I protect my forgejo instance from AI Web Crawlers by adastra22 on Monday December 22, 2025 @06:03AM (#46353159)
Why don’t they scrape once though?
[ Parent | on HN ]
Re:I protect my forgejo instance from AI Web Crawlers by blell on Monday December 22, 2025 @06:11AM (#46353202)
1) It may be out of date 2) Storing it costs money
[ Parent | on HN ]
Re:I protect my forgejo instance from AI Web Crawlers by adastra22 on Tuesday December 23, 2025 @05:59AM (#46364300)
Bandwidth costs more money.
[ Parent | on HN ]