Re:Hacker News bans Google and all other search engines
by Andys on Tuesday March 16, 2010 @01:29AM
(#1195101)
What are the details of crawl-delay? If its supposed to be "number of seconds between hits", for some sites, 1 hit per second may as well be "not crawled at all"..
[ Parent
| on HN ]
Re:Hacker News bans Google and all other search engines
by ComputerGuru on Tuesday March 16, 2010 @01:43AM
(#1195124)
Right, but his point is that at least this way you won't get delisted :)