News for hackers. Stuff that matters.

A comment on Robots.txt Disallow: 20 Years of Mistakes To Avoid.

Re:Robots.txt Disallow: 20 Years of Mistakes To Avoid by spaulo12 on Monday June 30, 2014 @01:51PM (#7966455)
In the past I've created an empty robots.txt just to keep the 404 errors out of my logs...
[ Parent | on HN ]

There are no comments on this comment yet.