News for hackers. Stuff that matters.
Re:Nordstrom's robots.txt by troels on Saturday February 28, 2015 @05:54PM (#9125357)
Yeah, the clue is in "Server: AkamaiGHost". The site is clearly behind some sort of anti-crawl protection which decides it doesn't like me. I think it would make sense to make an exception for exactly that file though.

There are no comments on this comment yet.