Nordstrom's robots.txt

Re:Nordstrom's robots.txt by woonord on Friday February 27, 2015 @06:04PM (#9121938)

Update! offending term we rank on "extra small teen porn" - offending page /sr/petite-extra-small. Robots updated (live tomorrow)! Asked Google to remove the URL.

[ Parent | on HN ]

Re:Nordstrom's robots.txt by temuze on Friday February 27, 2015 @06:09PM (#9121958)

So uh, how did you find out that you match for this phrase?

[ Parent | on HN ]

Re:Nordstrom's robots.txt by thephyber on Friday February 27, 2015 @06:34PM (#9122057)

Google Webmaster Tools shows you the search terms used to click thru to your website from Google Searches. I'm not sure if it still works, but referer URLs used to log search terms in the web access logs.

[ Parent | on HN ]

Re:Nordstrom's robots.txt by Buge on Friday February 27, 2015 @11:33PM (#9122796)

Probably looked at their logs and found a lot of visits to that URL.

[ Parent | on HN ]