Re:You Should Feed the Bots
by mcdeltat on Sunday October 26, 2025 @10:18PM
(#45716750)
Maybe a dumb question but what exactly is wrong with banning the IPs? Even if the bots get more IPs over time, surely storing a list of bans is cheaper than serving content? Is the worry that the bots will eventually cycle through so many IP ranges that you end up blocking legit users?
[ Parent
| on HN ]
It's often one IP (v4!) per one request. It's insane how many resources are being burned on this stupidity.
Part of the reason I did this is to get good numbers on how bad the problem is: A link maze is a great way to make otherwise very stealthy bots expose themselves.
Even if this is true how long can that be sustained before they start to be recycled? I bet the scrappers make a whole lot more requests than they have IPs
They are usually using residential IPs through SOCK5. I am not sure how they are getting these residential IPs but it is definitively suspicious.
So by blocking these IPs, you are blocking your users. (ie: in many coffeshops, I get the "IP Blocked" banner, my guess is that they are running software on unsuspecting users to route this traffic).
They use scammy providers like Bright Data[1] that let app authors embed their malware (for a compensation, I'm sure) which turns users' devices into crawler proxies.
> So by blocking these IPs, you are blocking your users.
There were 122 million residential internet connections in the US in 2024 so for an app with 1 million users the chance of affecting a single user is <1%.
[1] https://docs.fcc.gov/public/attachments/DOC-411463A1.pdf