Re:Creepy Crawlies
by cyberax on Sunday August 30, 2026 @03:09PM
(#49501793)
I have a small personal site with an exposed Git repository. I'm getting crawled from about 100000 residential IPs from several ASNs (mostly in Brazil). Each crawler makes a couple of requests per minute.
Anubis doesn't really help against this.