News for hackers. Stuff that matters.
Re:Hacker News bans Google and all other search engines by Andys on Tuesday March 16, 2010 @01:29AM (#1195101)
What are the details of crawl-delay? If its supposed to be "number of seconds between hits", for some sites, 1 hit per second may as well be "not crawled at all"..
Re:Hacker News bans Google and all other search engines by ComputerGuru on Tuesday March 16, 2010 @01:43AM (#1195124)
Right, but his point is that at least this way you won't get delisted :)