AI companies cause most of traffic on forums

Re:AI companies cause most of traffic on forums by petre on Thursday January 02, 2025 @08:46AM (#42574346)

You rate limit them and then block the abusers. Nginx allows rate limiting. You can then block them using fail2ban for an hour if they're rate limited 3 times. If they get blocked 5 times you can block them forever using the recidive jail.

I've had massive AI bot traffic from M$, blocked several IPs by adding manual entries into the recidive jail. If they come back and disregard robots.txt with disallow * I will run 'em through fail2ban.

[ Parent | on HN ]

Re:AI companies cause most of traffic on forums by herbst on Thursday January 02, 2025 @12:14PM (#42576295)

Whatever M$ was doing still baffles me. I still have several azure ranges in my blocklist because whatever this was appeared to change strategie once I implemented a ban method.

[ Parent | on HN ]

Re:AI companies cause most of traffic on forums by petre on Thursday January 02, 2025 @03:20PM (#42578388)

They were hammering our closed ticketing system for some reason. I blocked an entire C block and an individual IP. If needed I will not hesitate banning all their ranges, which means we won't get any mail from Azure, M$ office 365, since this is also our mail server. But scew'em, I'll do it anyway until someone notices, since it's clearly abuse.

[ Parent | on HN ]