News for hackers. Stuff that matters.
Re:Applebot, the web crawler for Apple by WorldWideWayne on Friday May 08, 2015 @10:56AM (#9512029)
There are some court cases that involve robots.txt but I'm not sure because there was a lot of reading to do and I was hoping someone who knew would just come along and summarize it for us :)

There are no comments on this comment yet.