News for hackers. Stuff that matters.
Re:The Definition of Evil: Microsoft's Search Wars Hurt Us All by blasdel on Monday November 23, 2009 @04:34PM (#958197)
Even if the spider doesn't respect robots.txt, the act of indexing is really obvious -- your webserver does not have to respond to their HTTP Requests if you don't want it to.

It's not as facile as 'lock your doors' -- instead just not answering when they knock.

There are no comments on this comment yet.