News for hackers. Stuff that matters.
Re:Ask HN: Why am I not permitted to crawl HN? by mooism2 on Monday May 20, 2013 @04:21AM (#5736527)
TIL robots.txt has a crawl-delay directive. Thank you.

There are no comments on this comment yet.