Re:Blunt Banhammer Blocks Basic Bots Ballistically
by joshblack1982 on Tuesday November 10, 2026 @08:06AM
(#50439760)
Huh neat. Guess I found it in the robots.txt because now it says "Payment Required - Access Denied. See https://shkspr.mobi/llms.txt"
Since my ISP has me behind CGNAT I wonder if anyone else on my same ISP is blocked. Hell I could keep restarting my router, getting assigned new IPs, and going to that URL to make sure everyone using the same provider is blocked. Could do it on my cellphone too to be extra thorough, since cell networks give many users the same IP with CGNAT.
Re:Blunt Banhammer Blocks Basic Bots Ballistically
by mzajc on Tuesday November 10, 2026 @06:47PM
(#50447938)
Two more potential problems with this:
- Someone might (maliciously or not) send a honeypot URL to a messaging service with URL embeds, try archiving it with IA or archive.today, try opening it in Google Translate, or use any other kind of manually-operated service that ignores robots.txt. This would prevent said service from working on the site entirely, unless you had whitelists.
- Someone might maliciously add images or such linking to a honeypot URL from their site to get people banned.
Not sure how likely these scenarios are or if they can be guarded against without giving the bots a loophole, but I wouldn't feel comfortable IP-banning visitors on my sites for sending GET requests to the wrong URL.
I have my own set of scripts that permanently bans IP addresses that have probed any non-service ports of my firewall.
Since my ISP has me behind CGNAT I wonder if anyone else on my same ISP is blocked. Hell I could keep restarting my router, getting assigned new IPs, and going to that URL to make sure everyone using the same provider is blocked. Could do it on my cellphone too to be extra thorough, since cell networks give many users the same IP with CGNAT.
That could be a problem for a major site - but I doubt your neighbours are going to miss reading my blog.
I'll probably set it up to remove IPs after a few weeks.
But, ultimately, I care more about my bandwidth than I do the collateral damage of a misidentified IP.
- Someone might (maliciously or not) send a honeypot URL to a messaging service with URL embeds, try archiving it with IA or archive.today, try opening it in Google Translate, or use any other kind of manually-operated service that ignores robots.txt. This would prevent said service from working on the site entirely, unless you had whitelists.
- Someone might maliciously add images or such linking to a honeypot URL from their site to get people banned.
Not sure how likely these scenarios are or if they can be guarded against without giving the bots a loophole, but I wouldn't feel comfortable IP-banning visitors on my sites for sending GET requests to the wrong URL.