Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by koakuma-chan on Tuesday March 25, 2025 @10:12PM
(#43478147)
Companies running those bots have more than enough resources
[ Parent
| on HN ]
Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by prisenco on Tuesday March 25, 2025 @10:17PM
(#43478170)
Nobody has unlimited resources. Everything is a cost-benefit analysis.
For highly valuable information, they might throw the GDP of a small country at scraping your site. But most information isn't worth that.
And there are a lot of bad actors who don't have the resources you're thinking of that are trying to compete with the big guys on a budget. This would cut them out of the equation.
Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by timewizard on Tuesday March 25, 2025 @10:42PM
(#43478327)
Make all websites intentionally waste energy as a strategy to defeat unscrupulous operators has negative costs and marginal benefits.
Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by noosphr on Tuesday March 25, 2025 @11:06PM
(#43478468)
Then use it to mine monero or similar.
The idea that you should pay for content shouldn't be an insane pipedream. It should be the default on the internet.
Maybe then we wouldn't be in the situation where getting new users is an existential threat to the majority of websites.
Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by prisenco on Tuesday March 25, 2025 @10:47PM
(#43478356)
Resources applied to prevent bad actors from degrading or destroying the commons has always been the cost of civilization.
Re:Devs say AI crawlers dominate traffic, forcing blocks on ...
by 01HNNWZ0MV43FF on Wednesday March 26, 2025 @01:36AM
(#43479167)
I'll add a link on my site recommending that visitors petition their elected officials for a pollution tax