News for hackers. Stuff that matters.

A comment on How to crawl a quarter billion webpages in 40 hours (2012). Parent comment

Re:How to crawl a quarter billion webpages in 40 hours (2012) by colechristensen on Thursday July 05, 2018 @11:01AM (#17463771)
Does that lack of restrictions include allowing me to decide who has access to things I make available?
[ Parent | on HN ]
Re:How to crawl a quarter billion webpages in 40 hours (2012) by anc84 on Friday July 06, 2018 @04:28AM (#17469847)
Nothing stops you from not serving those who you don't want to serve.
[ Parent | on HN ]