News for hackers. Stuff that matters.

A comment on Google's robots.txt. Parent comment

Re:Google's robots.txt by declan on Sunday February 23, 2014 @03:17AM (#7285311)
The operator of a crawler doesn't need to sign an agreement for the prohibition to be enforceable. See eBay v. Bidder's Edge. This was 14 years ago, folks.
[ Parent | on HN ]

There are no comments on this comment yet.