News for hackers. Stuff that matters.

A comment on AI companies cause most of traffic on forums. Parent comment

Re:AI companies cause most of traffic on forums by Capricorn2481 on Tuesday December 31, 2024 @03:12PM (#42561452)
I don't understand this. You don't have routes your users might need in robots.txt? This article is about bots accessing resources that other might use.
[ Parent | on HN ]
Re:AI companies cause most of traffic on forums by IncreasePosts on Tuesday December 31, 2024 @04:44PM (#42562087)
It seems better to put fake honeypot urls in robots.txt, and block any up that accesses those.
[ Parent | on HN ]
Re:AI companies cause most of traffic on forums by Capricorn2481 on Tuesday December 31, 2024 @05:32PM (#42562347)
Ah I see
[ Parent | on HN ]
Re:AI companies cause most of traffic on forums by trod1234 on Wednesday January 01, 2025 @06:50PM (#42570290)
Blocking will never work.

You need to impose cost. Set up QoS buckets, slow suspect connections down dramatically (almost to the point of timeout).

[ Parent | on HN ]