News for hackers. Stuff that matters.
Re:Ask HN: Relationship between HN and Scribd? by natrius on Sunday February 14, 2010 @08:11PM (#1125647)
This interpretation makes caching illegal and puts routing in a gray area. If not instructing against caching is implied consent to redistribute the content, then you're essentially agreeing with me.

robots.txt is indeed intended for crawling, but if it's there and you redistribute someone's content anyway, I'd consider it less defensible.

There are no comments on this comment yet.