News for hackers. Stuff that matters.

A comment on Google.com/killer-robots.txt. Parent comment

Re:Google.com/killer-robots.txt by sschueller on Friday July 04, 2014 @11:03AM (#7989014)
February 1994[1], so the 20 year anniversary of the robots.txt file.

[1]http://en.wikipedia.org/wiki/Robots_exclusion_standard

[ Parent | on HN ]

There are no comments on this comment yet.