Ask HN: Adaptive Spider Technologies?

Posted by throwavay452 on Wednesday August 03, 2011 @06:44PM
from the ask-hn dept.

Today a bot from 174.127.132.210 tried to copy information from my stealth-mode site (not linked to any domain name). This site was not supposed to be searchable and is known to a very limited number of people.

Any experience with such bots?

( 2 comments | on HN )

Re:Ask HN: Adaptive Spider Technologies? by tshtf on Wednesday August 03, 2011 @06:55PM (#2843568)

I've been blocking 174.127.132.0/22 for a while now.

It's generally not worth getting into an arms race against misbehaved bots, unless you're prepared to setup a honeypot or seriously analyze your access logs.

[ Parent | on HN ]

Re:Ask HN: Adaptive Spider Technologies? by throwavay452 on Wednesday August 03, 2011 @07:06PM (#2843611)

Thanks! Good to know it's common

[ Parent | on HN ]