News for hackers. Stuff that matters.
Re:Big Tech Has Become Too Powerful by caseysoftware on Saturday September 19, 2015 @02:32PM (#10245121)
So it's more out of ignorance than maliciousness. That makes sense.

Odds are the more troublesome scrapers are probably ignoring robots.txt anyway, so I wonder how useful it is.

There are no comments on this comment yet.