News for hackers. Stuff that matters.
Re:Nordstrom's robots.txt by CoreSet on Friday February 27, 2015 @04:51PM (#9121588)
I don't get it: is that disallow targeting some sort of porn spambot crawler?
Re:Nordstrom's robots.txt by jarin on Friday February 27, 2015 @04:55PM (#9121612)
They probably made a product with that slug by accident and are doing that to try to get it removed from search engine results.
Re:Nordstrom's robots.txt by meej on Friday February 27, 2015 @05:52PM (#9121891)
It makes me wonder if Nordstrom ever stocked the now-defunct Pornstar clothing brand.
Re:Nordstrom's robots.txt by Springtime on Friday February 27, 2015 @04:58PM (#9121628)
Disallow is used to exclude certain URLs from being crawled, in this case it looks like a search query as the store's search uses this pattern: http://shop.nordstrom.com/sr/query .
Re:Nordstrom's robots.txt by philip1209 on Friday February 27, 2015 @04:55PM (#9121617)
Maybe some mal-intended inbound links that they don't want to be associated with?
Re:Nordstrom's robots.txt by cmdrfred on Friday February 27, 2015 @06:56PM (#9122138)
Is this some sort of new terrible negative seo? Associate a target url for a bunch of negative things?
Re:Nordstrom's robots.txt by prokizzle on Friday February 27, 2015 @04:52PM (#9121592)
Maybe that was their intention, but that's not how it's done.